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EXECUTIVE  SUMARY 


'This  report  summarizes  the  results  of  the  Distributed  Instructional  System 
(PIS?  project.  The  project  was  carried  out  between  January,  1981  and  May,  1982 
'as  a  cooperative  effort  between  the  U.S.  Army  Training  Developments  Institute 
(TOI)  of  the  Array  Training  and  Doctrine  Command  (TRADOC) ,  the  Amy 
Communicative  Technology  Office  (ACPO)  of  the  U.S.  Amy  Ccmrrunications 
Electronics  Command  (CECCH) ,  the  U.S.  Amy  Air  Defense  School  (USAADS)  at  Ft. 
Bliss,  Texas,  and  WICAT  Systems  Incorporated  of  Orem,  Utah. 

The  DIS  consists  of  individual  student  workstations  connected  to  a  central 
storage  station  having  main  storage  capability.  Each  workstation  consists  of  a 
16-hit  WICAT  microcomputer  controlling  a  videodisc  player.  The  system  is 
operated  through  a  specially  designed  hand-held  keypad  and  can  provide 
ooabination  videodisc  and  computer-generated  displays.  The  DIS  system  is 
designed  to  simultaneously  deliver  computer-assisted  instruction  (CAI)  and  2- 
diaensional  (2-0)  simulation  over  a  broad  range  of  weapons  systems  and  trained 
tasks  to  operator  and  maintenance  personnel  simultaneously. 


The  present  project  was  the  third  and  culminating  phase  of  a  system  development 
effort  funded  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  for  the 
^first  two  phases.  Third  phase  funding  was  provided  by  TDI . 

The  goal  of  the  three  year  effort  was  the  development  of  a  low-cost  ccnputer- 
controlled  Interactive  videodisc  delivery  system  capable  of  providing  computer- 
assisted  instruction  and  2-D  simulation.  This  report  describes  the  system 
constructed  to  meet  that  goal,  as  well  as  the  products  created  and  validated  in 
a  demonstration  of  the  instructional  capabilities  of  the  system.  Section  2.0 
provides  a  brief  history  of  the  system  development  effort  and  its  goals. 

Section  3.0  describes  the  principles  that  guided  system  design  and  the  designs 
which  resulted.  Section  4.0  presents  the  work  plan,  schedule,  and  organization 
created  for  development  of  the  system,  and  Section  5.0  describes  the  results 
obtained.  Section  6.0  gives  a  set  of  recommendations  for  further  development 
of  the  concepts  developed  and  implemented  during  this  project. 

The  findings  of  the  project  were  that  100%  of  the  students  usi^g  the  DIS  were 
capable  of  completing  the  performance  test  problems,  whereas  only  25%  of  the 
students  not  receiving  the  training  were  able  to.  Moreover,  all  students 
completing  one  or  more  simulation  problems  in  addition  to  CAI  lessons  were  able 
to  oonplete  the  same  test  in  half  the  time.  Students  reported  enjoying  the 
lessons  and  simulation  problems  and  recommended  its  use  at  both  school  and 
field  sites.  Project  results  led  to  a  set  of  recommendations  for  improvement 
and  expansion  of  system  training: 

1.  Expansion  of  simulation  scope  to  include  all  I-HAWK  Integrated  Systems 
Checks. 

2.  Improvements  in  the  equipment  and  software  used  in  the  delivery 
system. 


■rar  4T, 


3.  Expansion  of  the  DIS  System  evaluation  to  cover  areas  not  approached 
this  evaluation. 

inproved  base  of  Instructional  Systems  Development  (ISD)  processes 
the  development  of  future  products. 
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the  teamwork  concept  employed  by  the  organizations  taking  part  in  this  project 
cannot  be  overemphasized  as  a  key  factor  in  its  success.  Especially  notable  in 
this  respect  were  the  efforts  of  USAADS  representatives  Mr.  Peter  J.  Baker  and 
CM2  Donald  Cooper,  whose  technical  and  liaison  activities  were  indispensable  at 
several  points  and  whose  dedication  to  the  task  was  far  beyond  the  usual 
effort.  Also,  Mr.  Bryan  Altman  of  TDI  served  a  critial  communication  and 
coordination  function  which  held  the  development  team  together  during  some  of 
the  more  difficult  decision-making  processes. 

Within  WICAT,  the  efforts  of  several  instructional  development  specialists, 
programmers  and  engineers  were  blended  into  the  creation  of  lesson  and 
simulation  designs.  In  addition  to  the  authors,  the  following  persons  were 
members  of  the  HANK  design  team: 

Dr.  A.  F.  O'Neal  -  responsible  for  conceptualizing  the  structure  and 
mechanism  of  CAI  programs  and  a  co-designer  of  the  simulation. 

Dr.  Deter  Fairweather  -  responsible  for  simulation  program  mechanism 
design. 

Dr.  J.  Olin  Campbell  -  a  co-designer  of  the  simulation. 

Dr.  David  M.  Tuttle  -  responsible  for  lesson  and  test  design  for 
instructional  materials  and  production  of  lessons. 

Mr.  Gary  Brown  -  programmer  during  early  stages  of  simulation  development 
and  responsible  for  many  basic  tools. 

Mr.  Dennis  Furse  -  programmer  for  the  lesson  driver  and  lesson  builder 
programs. 

In  addition  to  these  major  design  and  production  contributions,  the  efforts  of 
a  capable  staff  of  writers  and  programmers  was  also  inportant. 


DISTRIBUTED  INSTRUCTIONAL  SYSTEM  PROTECT 
FINAL  REPORT 

1.0  Introduction 

This  report  summarizes  the  results  of  the  Distributed  Instructional  System 
(DIS)  project.  The  project  was  carried  out  between  January,  1981  and  May,  1982 
as  a  cooperative  effort  between  the  U.S.  Arity  Training  Developments  Institute 
(TDI)  of  the  U.S.  Army  Training  and  Doctrine  Command,  the  Anty  Communicative 
Technology  Office  (ACTO)  of  the  U.S.  Arny  Cormunications-Electronics  Command 
(CEOCM) ,  the  U.S.  Amy  Air  Defense  School  (USAADS)  at  Ft.  Bliss,  Texas,  and 
WICAT  Systems  Incorporated  of  Orem,  Utah. 

The  DIS  consists  of  individual  student  workstations  connected  to  a  central 
storage  station  having  main  storage  capability.  Each  workstation  consists  of  a 
16-bit  WICAT  microcomputer  controlling  a  videodisc  player.  The  system  is 
operated  through  a  specially-designed  hand-held  keypad  and  can  provide 
combination  videodisc  and  computer-generated  displays.  The  DIS  system  is 
designed  to  simultaneously  deliver  computer-assisted  instruction  and  2- 
dimensional  (2-D)  simulation  over  a  broad  range  of  weapons  systems  and  trained 
tasks  to  operator  and  maintenance  personnel  simultaneously. 

The  present  project  was  the  third  and  culminating  phase  of  a  system  development 
effort  funded  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  for  the 
first  two  phases.  Third  phase  funding  was  provided  by  TDI. 

The  goal  of  the  three  year  effort  was  the  development  of  a  low-cost  computer- 
controlled  interactive  videodisc  delivery  system  capable  of  providing  computer- 
assisted  instruction  and  2-D  simulation.  This  report  describes  the  system 
.instructed  to  meet  that  goal  as  well  as  the  products  created  and  validated  in 
a  demonstration  of  the  instructional  capabilities  of  the  system.  Section  2.0 
provides  a  brief  history  of  the  system  development  effort  and  its  goals. 

Section  3.0  describes  the  principles  that  guided  system  design  and  the  designs 
which  resulted.  Section  5.0  presents  the  work  plan,  schedule,  and  organization 
created  for  development  of  the  system,  and  Section  5.0  describes  the  results 
obtained.  Section  6.0  gives  a  set  of  recommendations  for  further  development 
of  the  concepts  developed  and  implemented  during  this  project. 


2.0  Project  Origin  and  Goals 

This  section  provides  a  context  for  the  activities  of  this  Phase  III  project  by 
describing  work  completed  in  the  two  previous  system  development  phases. 

2.1  History 

In  August  1978,  the  Defense  Advanced  Research  Projects  Agency  (DARPA) 
contracted  with  WICAT  to  develop  the  concept  and  prototype  of  a  Distributed 
Instructional  System  (DIS) .  The  goal  was  a  low-cost  delivery  system  using 
computer-controlled  interactive  videodisc  as  a  medium,  for  instruction  and  2-D 
simulation,  for  both  field  and  school-based  military  training. 
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The  development  effort  beginning  with  design  studies  and  ending  with  a  fielded 
system  was  spread  over  three  independently-funded  project  phases.  The  first 
project  phase  was  to  be  a  study  and  design  phase  in  which  military  training 
needs  and  trends  were  examined.  These  needs  and  trends ,  once  identified,  were 
to  be  used  as  the  basis  for  creating  a  design  for  a  broadly  useful  and 
practical  military  training  delivery  system.  The  second  project  phase  was  to 
use  the  design  to  construct  a  prototype  system,  develop  sample  instructional 
materials  to  be  used  on  it,  and  run  preliminary  tests  on  those  materials.  The 
third  project  phase  was  to  be  the  full-scale  development  of  instruction  and  2 HD 
simulation  materials  followed  by  fielding  and  testing  of  the  system  in  a  school 
environment. 

Phases  I  and  II  were  accomplished  as  planned.  A  series  of  visits  were  made  by 
Dr.  C.  V.  Bunderson,  Dr.  J.  0.  Campbell  and  others  to  a  number  of  training 
sites  within  the  U.S.  Army,  as  well  as  private  contractors  involved  in  military 
training  development.  Trend  data  was  gathered  from  these  sources  as  well  as 
data  on  the  current  status  of  military  training  programs,  and  the  results  were 
summarized  in  a  report  entitled,  "A  Causal  Framework  to  Describe  Training  Needs 
and  Requirements  in  the  Department  of  Defense."  The  findings  were  that  a 
powerful  technology  base  was  requisite  within  military  training  to  satisfy  the 
demands  of  increasingly  sophisticated  weapons  training,  lowering  budgets,  and 
fluctuations  in  trainee  quality.  The  need  for  a  Distributed  Instructional 
System  (DIS)  for  the  delivery  of  training  was  established  in  the  report. 

Concurrent  with  this  study  of  training  needs,  a  second  study  of  simulation 
capabilities  and  requirements  was  completed  after  a  thorough  review  of  the 
literature  by  Dr.  J.  0.  Campbell.  A  resulting  report,  entitled  "Intelligent 
simulation  and  Job-Aiding  on  a  Distributed  Instructional  System, "  built  on  the 
recommendations  of  the  needs  and  requirements  study.  It  catalogued  the 
features  for  simulation  and  job-aiding  which  are  possible  using  a  system  of 
DIS-like  design. 

At  this  point,  the  design  of  a  DIS  system  was  complete,  and  the  development  of 
a  DIS  prototype  was  underway.  A  description  of  the  system  as  it  was  then 
conceived  is  presented  in  Appendix  A.  The  description  is  essentially  unchanged 
to  the  present  day  except  for  the  use  of  coaxial  cable  rather  than  fiber  optic 
links  between  stations  in  the  present  system.  Creating  the  system  prototype 
included  construction  of  individual  workstations  and  their  operating  software 
and  creation  of  network  sotware  for  linking  workstations  to  the  central  storage 
station. 

The  Courseware  Design  System  (CDS) ,  a  high-level  authoring  language,  was  used 
in  this  prototype  system  for  lesson  development.  Eleven  lessons  of 
instruction,  comprising  three  videodisc  sides,  were  developed  for  DIS.  These 
lessons  were  to  be  the  vehicles  for  an  initial  evaluation  of  the  system,  its 
programs,  and  CDS. 

To  guide  that  evaluation  and  to  provide  tools  for  assessing  the  outcome,  two 
additional  reports  were  produced.  The  first  of  these  developed  the  important 
concept  of  productivity,  in  which  the  instiuctional  process  was  looked  upon  as 
an  event  requiring  a  given  level  of  resources  and  having  a  measurable  level  of 
outcome.  The  ratio  of  these  was  termed  'die  productivity  of  the  system.  The 
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report  containing  this  concept,  authored  by  Dr.  J.  Olsen,  was  entitled  "Learner 
and  Learning  System  Productivity"  and  contained  an  explanation  of  the  concept, 
a  review  of  the  literature  on  productivity,  and  an  evaluation  plan  for  an 
evaluation  intended  to  assess  DIS  productivity.  The  evaluation  plan  called  for 
measurement  of  several  variables  in  an  assessment  of  productivity  attained 
during  use  of  the  DIS. 

The  second  report  written  to  provide  a  foundation  for  the  evaluation  was 
entitled  "A  Cost/Benefits  Analysis  for  Implementing  the  Distributed 
Instructional  System."  In  this  report  Mr.  T.  Compton  and  Dr.  G.  Kearsley 
proposed  a  unique  methodology  for  projecting  the  cost  implications  of 
implementing  a  family  of  training  devices  within  a  training  program,  each  with 
an  expected  degree  of  transfer  to  real  job  performance  ability.  The  important 
feature  of  this  cost  mechanism  was  the  verifiability  of  the  outcomes  of  its 
projections  and  its  potential  use,  given  accumulation  of  an  adequate  data  base, 
for  projecting  the  cost-optimal  group  of  training  delivery  systems  for 
virtually  any  training  project,  given  the  levels  of  proficiency  desired. 

By  the  end  of  Phases  I  and  II,  the  DIS  system  had  been  installed  at  the  USAADS 
Ft.  Bliss  test  site  ready  for  an  initial  evaluation  of  the  instructional 
materials  in  the  context  of  MOS  24E10  Integrated  Systems  Check  training.  Data 
collection  was  begun,  but  before  this  evaluation  could  be  completed,  a 
redirection  of  effort  initiated  preparations  for  an  enlarged  test  and 
evaluation  effort  under  the  sponsorship  of  TDI.  This  test  was  to  include 
evaluation  of  the  2-D  simulation,  which  at  the  end  of  Phase  II  had  been 
partially  designed  but  not  developed .  Evaluation  was  also  to  cover  additional 
instructional  materials  on  videodisc  to  supplement  those  already  created.  A 
contract  awarded  to  WICAT  by  TDI  in  January,  1981  established  the  tasks  for 
Phase  III  and  the  work  to  be  accomplished. 

3.0  Approach  and  Design  Rationale 

A  design  for  the  DIS  was  created  based  on  the  design  options  and  studies  from 
Phases  I  and  II.  This  design  was  a  composite  of  the  hardware,  software,  and 
courseware  designs  fitted  to  the  specific  training  needs  of  I-HAWK  training 
course  at  USAADS,  Fort  Bliss,  Texas. 

Before  the  design  process  began,  it  was  known  that  in  order  to  fill  the 
military  training  needs  identified,  the  system  would  have  to  have  certain  key 
qualities,  such  as: 

1.  Convenient  size,  peripheral,  and  environment  requirements.  If  the 
system  was  to  be  usable  in  a  variety  of  applications,  its  demands  for 
peripheral  equipment,  environment  conditioning,  and  power  supply  had 
to  be  kept  to  a  minimum  and  had  to  be  standard  and  widely  available. 

2.  Lew  acquisition  and  maintenance  costs.  The  costs  for  the  system  and 
its  programs  had  to  be  low  enough  to  permit  widespread  use. 

3.  High  fidelity.  The  displays  and  interactions  of  the  system  had  to 
have  as  high  a  fidelity  as  possible  to  real  job  performance  standards. 
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IWs  refers  to  the  fidelity  that  problems  would  require  of  the 
student's  mental  processes,  as  contrasted  with  the  fidelity  possessed 
by  real  equipment.  The  pitfalls  of  equating  fidelity  of  "touch"  and 
"feel"  with  fidelity  of  mental  process  were  recognized.  Careful 
design  in  favor  of  both  was  accomplished  where  possible,  but  more 
importantly  in  favor  of  behavioral  fidelity. 


4.  Maintainability.  Not  only  did  the  system  equipment  need  to  be  easily 
maintained,  but  the  software  programs  administering  instruction  and 
simulation  needed  to  be  field-modifiable  to  the  extent  possible. 

3.1  System  Configuration 


The  system  description  contained  in  Appendix  A  has  already  been  referred  to.  A 
■ore  detailed  description  of  system  equipment  is  given  below. 


following  equipment: 


Each  of  the  four  workstations  installed  consists  of  the 


1  -  workstation  computer,  WICAT  Model  100-ET 
5  -  circuit  boards 

1  -  CPU  board 

2  -  256K  memory  boards 

1  -  workstation  communications  board 
1  -  videodisc  interface  board 
1  -  Panasonic  13"  color  TV  monitor 
1  -  Keytronics  keyboard 
1  -  MCA  DiscoVision  Videodisc  Player, 

Model  PR-7820 


Each  workstation  was  also  equipped  with  a  custom-made  hand-held  keypad 
containing  a  set  of  special  instruction/simulation  control  keys.  These  keys 
are  identical  to  the  special  keys  provided  on  the  Keytronics  keyboard,  as 
described  in  later  sections  of  this  report. 


The  storage  station  was  connected  through  a 
shielded  cable  to  each  of  the  workstations.  Storage  station  equipment 
installed  consists  of  the  following: 


1  -  storage  station  computer,  WICAT  Model  100 
5  -  circuit  boards 
1  -  CPU  board 
1  -  256K  memory  board 
1  -  storage  station  communications  board 
1  -  Marksman  disk  interface  board 
1  -  floppy  disk  controller  board 
1  -  Century  Data  Intelligent  Marksman  14"  Winchester  disk  drive 
with  integral  controller  (20  megabytes) 

1  -  Data  Electronics  high-density  cartridge  magnetic  tape  drive 
1  -  Televideo  TV1-912C  CRT  terminal 
1  -  Control  Data  dual  71/2"  floppy  disk  unit 
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A  networking  program  installed  on  the  storage  station  enabled  storage  station 
cxxmunication  with  the  four  workstations  for  the  purpose  of  supplying  lesson  or 
simulation  data  and  programs  for  use  by  the  workstations.  The  mode  of 
operation  within  this  system  is  different  from  the  standard  time-shared  system. 
Individual  workstations,  having  512K  of  memory  are  capable  of  downloading 
entire  programs  and  data  bases  for  independent  processing  rather  than  having  to 
communicate  continually  with  the  central  computer.  Access  from  the  workstation 
to  the  storage  station  is  kept  to  a  minimum,  and  intervals  between 
communications  may  be  as  long  as  45  minutes  to  1  1/2  hours.  The  exceptions  to 
this  are  the  lesson  exam  process  and  the  logon  process,  in  which  interactions 
are  relatively  intense  for  short  periods  of  time.  During  the  exam  process, 
communications  take  place  at  an  estimated  median  value  of  every  2  minutes. 

3.2  Program  Design 

The  instruction  and  simulation  programs  consist  of  a  family  of  programs 
embedded  in  a  matrix  of  control  and  administrative  programs.  Table  1  on  page  6 
lists  the  programs,  and  Figure  1  on  page  7  shows  the  hierarchical  relation  of 
the  programs  to  each  other. 

Note  in  Figure  1  on  page  7  that  administrative  programs  are  handled  under  a 
separate  control  program  (Terminal  Control  Program)  to  prevent  student  access 
to  administrative  data.  Lesson  and  Simulation  Maintenance  programs  have  been 
separated  from  all  other  programs  to  avoid  their  use  by  unauthorized  persons. 
The  Lesson  Maintenance  program  in  particular  has  been  engineered  for  easy  use 
and  modification  of  lessons.  That  function,  however,  is  carefully  controlled 
and  limited  to  only  certain  users  to  avoid  haphazard,  unauthorized  changes  to 
lessons  and  potential  lesson  tanpering. 

Figure  2  on  page  8  shows  the  availability  of  programs  at  the  workstation  and 
the  storage  station.  It  is  not  possible  for  the  workstation  to  run  storage 
station  programs,  though  the  opposite  is  true.  This  provides  further 
protection  from  students  tampering  with  the  system.  The  Exam  Maintenance 
program  is  available  at  the  workstation  so  that  instructors  may  see  how  test 
items  will  look  on  the  workstation  screen.  However,  access  to  the  program  is 
only  available  to  those  logging  on  as  instructors. 

The  decision  process  during  log-on  shown  in  Figure  3  on  page  9.  The  first 
interaction  is  a  request  for  the  number  of  the  videodisc  side  playing  and  a 
press  of  the  NEXT  key.  This  allows  the  computer  to  set  up  its  input  files  and 
adjust  to  the  use  of  either  the  hand-held  keypad  or  the  Keytronics  keyboard. 

The  title  page  then  requests  a  user  number.  This  title  page  is  a  videodisc 
frame,  as  are  all  of  the  subsequent  menus.  If  a  user  is  logging  on  for  the 
first  time  to  the  system,  a  system  introduction  lesson  is  administered  by  the 
lesson  driver.  This  lesson  familiarizes  the  user  with  the  cursor  and  return 
keys  for  entering  options. 

The  system  menu  is  presented,  allowing  the  user  to  request  either  simulation  or 
instruction.  On  this  and  all  subsequent  menus  the  user  may  select  to  log-off. 
If  the  user  is  an  instructor  or  manager,  the  system  recognizes  this  and 
presents  a  special  instructor  menu.  This  allows  not  only  all  student  choices 
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Table  1 


DIS  System  Programs 


Program 


Description 


Lesson  Driver 

Runs  all  sixteen  of  the  troubleshooting  exercise 
lessons  and  all  CAI  lessons. 

Lesson  Builder 

Maintains  the  sixteen  troubleshooting  exercises  and 
all  CM  lessons. 

Exam  Maintenance 

Allows  instructors  to  maintain  the  pool  of  exam 
questions. 

Exam  Driver 

Administers  exams  and  stores  scores. 

Simulation  Driver 

Runs  the  simulation  of  HAWK  circuits. 

Simulation  Maintenance 

Maintains  the  HAWK  component  description  files ,  the 
rules  files  that  control  the  simulation,  and  the 
data  files  required  for  the  simulation. 

User  Registration 

Allows  managers  to  add  new  users  to  the  system  or  to 
delete  old  ones. 

Logon 

Logs  a  student  onto  a  workstation  and  allows  him/her 
to  choose  instruction  or  simulation. 

Simulation  Problem 
Generator 

Allows  the  instructors  to  create  new  simulation 
problems  for  students. 

Report  Generator 


Produces  reports  for  instructors  on  student  progress 
and  on  lessons  and  simulation  problems. 


but  also  the  use  of  the  Exam  Maintenance  program  and  direct  access  to  all 
system  introductory  lessons  (Introductions  to  the  System,  the  Instruction,  and 
the  Simulation) . 

If  instruction  is  selected  from  the  system  menu,  a  check  is  made  to  determine 
if  the  user  has  selected  instruction  before.  If  not,  an  instruction 
introduction  lesson  is  presented  by  the  lesson  driver  prior  to  the  presentation 
of  a  unit  menu.  This  introduction  lesson  demonstrates  the  use  of  the  special 
instruction  control  keys.  The  unit  menu  lists  the  units  available  for  study; 
and  when  a  unit  is  chosen,  the  appropriate  lesson  menu  is  presented.  This 
lists  the  lessons  within  a  unit.  When  a  lesson  is  chosen,  the  lesson  pre-menu 
is  presented  which  allows  the  user  to  enter  a  lesson  in  instruction  mode,  in 
review  mode,  or  to  take  the  lesson  exam. 

If  the  user  chooses  simulation  from  the  system  menu,  the  system  determines 
whether  it  is  the  user's  first  use  of  the  simulation  option.  If  so,  it 
automatically  presents  a  simulation  introduction  lesson.  This  lesson 
demonstrates  the  use  of  the  simulation  control  keys  on  the  hand-held  keypad. 
Following  the  simulation  introduction,  a  simulation  menu  is  presented  which 
lists  the  problems  assigned  to  the  user.  The  user  may  select  a  simulation 
problem  from  this  menu.  It  is  possible  for  the  instructor  to  both  assign  and 
disable  problems  from  this  menu  and  thus  constrain  the  choice  of  simulation 
problems  given  the  user.  Once  the  simulation  problem  has  been  chosen,  the 
simulation  driver  program  is  called,  and  the  simulation  problem  is 
administered. 

3.2.1.  Simulation  Surface  Design 

The  design  of  the  2-0  simulation  began  with  the  definition  of  its  scope.  It 
was  to  simulate  a  portion  of  the  I-HKWK  Integrated  Systems  Checks  (ISC) .  The 
missile  system  is  composed  of  several  major  operating  components  (e.g. ,  four 
types  of  radar,  one  Battery  Control  Central,  three  Launchers,  etc.) .  Since  the 
missile  is  designed  for  use  in  a  mo ve-and -shoot  environment,  quick  set-up  and 
calibration  is  an  absolute  necessity.  1316  ISCs  are  check  procedures  which 
allow  rapid  system  calibration  and  testing  after  set-up. 

During  the  performance  of  the  ISC,  MOS  24E10  maintenance  personnel  and 
personnel  from  several  other  MDS  groups  rust  be  prepared  to  troubleshoot  and 
correct  faults  as  they  occur.  The  specific  circuits  selected  for  simulation 
were  those  involved  in  the  4-Parameter  Test  of  the  HIPIR/LCHR  Check,  a  test  of 
the  missile  system's  ability  to  detect,  lock  on,  track,  and  fire  at  a 
computer-generated  test  target.  The  4-Parameter  Test  is  a  comprehensive  test 
of  most  system  functions,  and  the  faults  it  uncovers  require  maintenance 
personnel  to  troubleshoot  a  broad  range  of  malfunctions  of  the  missile  system 
to  isolate  problems  to  major  end  items. 

The  2D  simulation  was  developed  to  follow  a  sequence  of  CAI  lessons  containing 
a  necessary  foundation  of  knowledge  and  skills  for  troubleshooting. 

The  surface  design  (user  interface)  of  the  simulation  is  intended  to  give  the 
student  free  play  on  the  simulated  circuits  without  direction  or  correction. 
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while  providing  realistic  visual  and  decision-making  interactions  which 
require  the  student  to  deal  with  the  spatial  as  well  as  the  electronic 
realities  of  the  I-HAWK  System  Battery  Control  Central  (BCC) . 

Control  over  the  simulation  is  exercised  by  a  set  of  simulation  control  keys 
available  on  both  the  hand-held  keypad  and  the  keyboard.  A  listing  of  these 
keys  and  their  functions  provides  the  "feel"  of  the  simulation  interactions. 

gflTM-TH  Key  —  Allows  the  user  to  zoom-in  to  take  a  close  look  at  a 
chassis,  a  meter,  a  radar  screen,  a  switch,  a  lamp,  or  a  panel.  Zoom-in 
to  109  different  BCC  locations  is  accomplished  by  altering  the  location 
nunher  (given  to  the  user  on  the  locator  charts  provided  at  each  work¬ 
station)  and  pressing  ZOCM-IN.  Upon  zooming  in,  the  user  sees  the 
component  in  its  present  state  as  calculated  by  the  circuit  model 
(described  later) .  If  the  location  is  a  lamp  and  the  circuit  model 
calculates  that  the  lamp  is  on,  the  zoom- in  goes  to  a  visual  of  the 
illuminated  lamp.  After  this  fashion,  meters  show  accurate  readings  and 
motion  when  zoomed- in  to,  and  controls  are  shown  int  he  settings  the 
circuit  model  understands  them  to  be  in. 


ZQQM-(XJT  Key  —  Allows  the  user  to  "back  off"  from  the  component  last 
zoomed  to.  If  the  user  is  viewing  a  control,  the  first  ZOCM-OUT  press 
changes  the  visual  to  the  control's  general  panel  area.  Subsequent  ZOCM- 
OOTs  will  bring  up  photographic  or  art  views  of  the  console,  the  wall,  and 
the  BCC  overhead  (bird's-eye  view) .  By  pressing  ZOCM-OUT,  the  student  can 
maintain  orientation  within  the  BCC  and  view  the  major  landmarks  in  the 
area  of  most  immediate  operations. 

HMK  Bey  —  Allows  the  user  to  obtain  the  name  of  the  component,  control, 
lamp,  etc.  being  viewed  at  any  given  moment.  This  feature  is  especially 
useful  for  confirming  the  identity  of  a  chassis  which  has  been  zoomed  to 
for  measurement  purposes.  Also,  since  many  controls  look  similar  to  the 
closest  zoom-in,  the  NAME  feature  allows  the  user  to  confirm  that  the 
correct  meter  or  lamp  is  being  observed. 

wnrflTrm  Kay  —  is  the  user's  gate  behind  the  panels.  Measurement 
routines  require  the  user  to  zoom-in  to  the  component  being  measured, 
whether  a  terminal  board  or  a  jack.  Qi  certain  zoom-in  frames,  the  BKSD 
legend  on  the  screen  tells  the  user  that  pressing  BACKSIDE  will  open  the 
panel  to  an  interior  view-  Using  the  ZOCM-IN  key  from  that  point  then 
allows  the  student  to  explore  or  to  zoom  directly  to  a  measurable 
oonponent. 

lay  —  Allows  the  user  to  change  the  setting  of  any  of  the  settable 
controls.  Pressing  SET  adds  to  the  display  a  group  of  setting  option 
boxes  and  a  cursor.  When  the  cursor  is  placed  on  one  of  the  boxes  and 
ENTER  is  pressed,  the  display  changes  to  show  the  control  in  that  new 
setting.  At  the  same  time,  a  message  "STANDBY  -  CALCULATING"  indicates  to 
the  user  that  new  values  for  all  circuit  components  are  being  computed  by 
the  circuit  model. 
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***gr  Maw  —  Allows  the  user  to  run  two  diagnostic  tests  from  the  ISCs.  If 
the  user  selects  one  of  the  tests  (the  4-Parameter  Test) ,  successful 
running  of  the  test  after  correction  of  the  fault  terminates  the  problem 
and  congratulates  the  student.  If  a  test  is  selected  but  not  run,  the 
simulator  remembers  and  asks,  after  a  measured  delay,  if  the  test  is  going 
to  be  run. 

HTJffftr  fry  —  Allows  the  user  to  take  measurements  at  over  600 
measurement  points.  Measurements  may  use  the  multimeter,  digital 
voltmeter,  or  oscilloscope.  Measurement  types  include  volts  DC,  (negative 
and  positive),  volts  AC,  ohms,  waveforms,  serial  words,  and  volts  DC 
return.  The  user  must  zoan  to  the  area  of  the  measurement  point  before 
placing  probes,  either  at  an  appropriate  ground  or  on  a  measurable  point. 
Illegal  measurement  points  are  detected  and  denied  the  user.  In  the  case 
of  continuity  tests  (ohm  measurements)  and  AC  voltage  measurements 
involving  3-wire  synchro  circuits,  illegal  measurement  point  pairing  is 
detected  and  disallowed.  When  legal  points  have  been  selected,  the 
measurement  readout  is  given  to  the  user  as  a  display  of  the  face  of  a 
meter,  giving  the  reading  as  the  student  would  normally  receive  it  under 
field  conditions. 


HffiMR/HEPf-aPE  —  Allows  the  user  to  make  repairs  of  parts, 
replacements,  or  substitutions  to  effect  a  correction  of  the  fault. 

The  user  zooms  in  to  the  component  to  be  repaired  and  designates  the 
component  by  name.  In  some  cases,  the  problem  exists  in  circuits  outside 
the  BCC  which  the  24E10  is  not  responsible  for.  In  such  problems,  the 
REPAIR/REPLACE  function  allows  the  faulty  end  item  (for  instance,  the 
radar)  to  be  identified.  On  correct  identification,  the  problem  is 
terminated.  If  the  repairs  made  are  unnecessary  of  incorrect,  the 
simulation  accepts  them  without  comment  and  continues  the  problem.  If 
repairs  are  made  correctly,  the  student  is  still  obligated  to  run  the 
diagnostic  test  to  demonstrate  that  the  system  is  functioning  properly,  at 
which  time  the  problem  is  terminated. 

HK)  HU  Key  —  Allows  the  user  to  specify  a  control  at  another  end  item 
of  the  I-HNWK  battery  (for  instance,  the  radar)  to  be  set.  In  addition, 
readings  of  indicators  at  other  end  items  nay  be  checked,  and  control 
positions  may  be  determined. 

PPM  Key  —  Allows  the  user  to  recover  from  mistakes  about  to  be  made,  but 
not  from  mistakes  already  made.  Pressing  OOPS  allows  the  user  to 
terminate  a  measurement,  a  repair,  or  a  test,  or  to  back  out  of  an  end 
item  inquiry. 

Figure  4  on  page  13  presents  a  flow  diagram  of  the  simulation  process  from  the 
user's  point  of  view. 

As  the  user  chooses  simulation  from  the  system  menu,  the  first  encountered 
event  is  a  list  of  simulation  problems  assigned  to  the  student.  The  user 
selects  a  problem  and  the  simulator  begins  an  initialization  routine  in  which 
the  circuit  model  calculates  the  appropriate  circuit  values  for  the  problem 
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Figure  4 

Simulation  Process  Flow 
(user-view) 
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specified.  The  circuit  model  is  a  part  of  the  simulation  program  which 
calculates  a  value  for  each  of  the  1000  components  in  the  simulated  circuits. 
Calculation  is  accomplished  using  a  set  of  rules  which  contains  a  rule  for  each 

ccnponent. 

During  problem  initialization  the  user  is  advised  of  manuals  which  may  be  used 
during  the  simulation  and  a  countdown  process  indicates  the  number  of  seconds 
before  the  problem  will  be  started.  When  the  system  has  initialized  the 
circuit  model,  a  list  of  the  problem  symptoms  is  presented  to  the  user,  and  the 
user  is  notified  of  the  criteria  for  solving  the  problem. 

From  this  point  the  user  is  branched  to  a  view  of  the  BCC  overhead,  and  the 
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system  does  nothing  except  that  which  the  user  requests  through  the  simulation 
control  keys.  The  simulator  is  capable  of  detecting  when  a  solution  has  been 
reached,  and  upon  correct  solution  of  the  problem  the  user  is  presented  with  a 
set  of  scores  which  represent  the  performance  during  the  problem.  At  this 
point  the  user  is  invited  to  inspect  the  record  accumulated  during  the  problem, 
including  the  total  amount  of  time  elapsed  for  problem  solution,  the  total 
amount  of  "repair  time"  (the  amount  of  time  the  problem  would  have  taken  in  the 
real  world  due  to  replacements  and  repairs) ,  the  number  of  switches  set,  the 
number  of  indicators  read,  the  number  of  tests  run,  the  number  of  measurements 
taken,  and  the  number  of  repairs  or  replacements  made.  Following  this  score 
report,  the  user  is  returned  to  the  system  menu. 


3.2.2  Simulation  Internal  Design 

Hie  operation  of  the  simulation  is  accomplished  by  a  suite  of  four  main 
programs  operating  on  a  set  of  data  files.  The  simulation  programs  and  their 
functions  are  listed  below. 


SIMAIKP  —  Hie  SIMAINT  program  is  used  to  create  and  maintain  the 
complete  component  data  base  used  by  the  simulation  program.  Each 
component,  depending  on  its  type,  has  the  potential  of  requiring 
several  different  items  of  data  which  describe  its  operation 
characteristics.  Data  on  those  characteristics  was  initially  gathered 
using  a  component  description  form.  An  exanple  of  this  form  is  found 
in  Figure  5  on  page  15. 

SIMDEF  —  Hie  SIMDEF  program  translates  circuit  equations  from  the  source 
version,  which  is  human  readable,  to  a  form  that  can  be  directly 
calculated  by  the  simulation  program.  One  of  the  means  used  to  achieve 
speed  within  the  simulation  program  was  to  perform  calculations  using 
the  fastest  mode  possible.  Hiis  program  places  the  rules  in  the 
appropriate  form  for  those  calculations. 

SIMD  —  Hie  SIMD  program  is  used  to  generate  data  files  which  are  actually 
used  by  the  simulation  program.  Hie  SIMD  program  works  on  the 
component  data  base  to  create  the  files  used  when  any  of  the  special 
simulation  functions  is  invoked  such  as  the  SET  function,  or  the 
MEASURE  function. 
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Name _ 

System  location  xx 
Troubleshooting  technique 

Breakable _  Repairable 

Failure  Nodes 


Circ.  Diagr.  Locat 
Funct.  Diagr.  Locat 
Type  Class  xxx 

System  function  group _ 

_  Difficulty  level _ 

_  Measurable _  Settable _ 


Repair  type  _ 

Replacement  time  . 
Substitution  time 


Repair  time _ 

Replacement  unit_ 
Substitution  unit. 


Measurement  instrument _ 

Type  of  signal  _ 

Chassis  rear  measurement  point? 

Setting  time _ 

Initial  setting  _ 

Rule: 


Measurement  time. 
Danger  point  _ 


Number  of  setting  states  . 
Alternate  initial  setting 


Figure  5 

Component  Description  Page 


SIMDKVR  —  The  SIMDRVR  program  is  the  control  program  for  the  simulation 
and  drives  the  execution  of  the  simulation  control  routines  using  the 
data  files  created  by  SIMD. 

In  order  to  run  the  simulation,  there  mist  be  several  data  files  other  than  the 
component  rules  file.  Most  of  these  are  created  by  the  SIMD  program.  The 
major  files  and  their  purposes  are: 

Component  File  —  This  file  contains  the  names  of  all  components  in  the 
simulation  plus  the  simulation  variable  number  of  the  component  and 
data  regarding  whether  the  component  is  breakable,  repairable, 
settable ,  and  measurable. 

Measure  File  —  This  file  contains  information  for  all  components  that 
can  be  measured.  The  data  in  this  file  concerns  the  signal  type  at  a 
given  component,  the  instrument  which  can  be  used  for  measurement  of 
the  signal  at  that  point,  identification  of  whether  the  point  is  one  of 
a  triplet  of  measurement  points  as  in  synchro  wire  measurements,  and  a 
continuity  list  number  for  dims  measurement. 

Hie  data  in  these  two  files  are  not  used  as  frequently  by  the  simulation 
program  as  other  files  which  will  be  described.  When  data  from  these  files  is 
needed,  it  is  read  from  the  storage  station  memory  over  the  network  using  a 
random  access  KSAM  file  routine. 

The  files  which  are  listed  below  are  also  simulation  data  files  which  are  read 
into  the  memory  of  the  workstation  at  the  time  the  simulation  program  is  loaded 
to  the  workstation.  These  files  provide  fast  access  to  the  most  frequently 
used  simulation  data.  They  are  especially  structured  to  provide  fast  access  to 
to  the  data  and  also  to  minimize  memory  use.  These  files  are  a  second  factor 
used  to  minimize  calculation  and  response  time  to  the  user. 

Pointer  File  —  This  file  contains  a  set  of  pointers  to  records  in  data 
files  and  is  the  preliminary  means  the  simulation  uses  to  determine 
components  that  can  be  set,  chassis  from  which  measurements  can  be 
taken,  components  which  can  be  zoomed- in  to  or  zoomed-out  from, 
components  which  can  be  named  and  components  which  can  be  used  as  a 
gate  to  the  backside.  The  simulation  program  uses  this  pointer  file  as 
the  first  source  of  information  when  a  student  issues  a  simulation 
control  key  command.  If  the  necessary  data  to  execute  the  control 
function  is  not  available  in  the  pointer  file,  the  simulation  program 
knows  that  an  illegal  command  has  been  issued  and  responds 
appropriately. 

Primary  File  —  The  Primary  File  contains  a  record  giving  the  specific 
videodisc  frame  data  and  other  required  functional  data  for  each 
component  that  can  be  set,  zoomed- in  to,  zoomed-out  from,  measured,  or 
used  as  a  backside  gate.  Records  in  this  file  consist  of  a  pointer  to 
the  appropriate  individual  data  files. 
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Measure  Data  File  —  Hus  file  contains  a  list  of  components  by  name  and 
variable  number  that  can  be  measured  from  each  chassis. 

Set  Data  File  —  This  file  contains  data  for  each  settable  component  which 
allows  the  user  to  set  controls  and  manipulate  handwheels.  The  data 
contained  in  this  file  includes  videodisc  frame  data  and  setting 
position  data. 

Zoom- In  Data  File  —  This  file  contains  data  for  each  component  that  can 
be  zoomed- in  to.  The  data  contained  in  this  file  includes  the  number 
of  states  the  component  may  be  found  in  and  the  videodisc  frame  number 
to  be  accessed  for  each  state. 

Name  Data  File  —  This  file  contains  the  name  of  each  component  that  can 
be  zoomed- in  to. 

Ohmlist  Data  File  —  This  file  contains  lists  of  components  which  are 
chained  together  in  continuity  lists  for  the  purpose  of  enabling  ohms 
measurements.  Since  the  rule  structure  is  incapable  of  identifying 
continuity  chains  within  the  circuits,  this  file  is  necessary  to 
identify  legal  dim  measurement  pairings. 

The  circuit  model,  which  consists  of  the  complete  set  of  component  rules  and 
the  simulation  driver  program  which  acts  upon  them,  models  a  total  of  800 
equations  for  a  total  of  1050  components.  Many  of  the  equations  perform  a 
simple  assignment  from  the  output  of  the  previous  component  to  the  input  of  the 
following  component  in  the  circuit.  Even  in  the  simplest  case  of  equation,  a 
separate  rule  must  be  given  for  each  component  because  each  component  can  be 
broken.  The  existence  of  a  rule  enables  the  effect  of  breakage  on  the 
individual  components  and  subsequent  circuit  components  to  be  calculated. 

Secondly,  each  component  is  potentially  a  me asur ament  point  and  the  value  for 
measurements  must  be  calculated  and  assigned  to  that  point  so  that  when 
measurement  takes  place,  the  appropriate  value  may  be  reported.  The  circuit 
rules  for  many  components  do  change  the  input  significantly  in  their 
calculation  of  output  values.  In  some  cases,  notably  in  synchro  circuits,  a 
single  input  signal  enters  a  component  to  result  in  three  mathematically- 
balanced  output  signals.  The  reverse  is  also  true  at  certain  components  where 
three  signals  converge  on  a  single  component  and  must  be  converted  into  a 
single  resultant  signal. 

Special  challenges  during  the  development  of  the  2-D  simulation  programs 
included  the  variety  of  signal  types  modeled  by  the  simulation  (including  DC 
voltages,  both  positive  and  negative,  DC  return  circuits,  varying  AC  voltages 
and  wave  forms) .  In  sane  cases,  notably  wave  form  signals,  signal 
characteristics  had  to  be  divided  among  several  "ghost"  components  which  could 
be  broken  independently  to  produce  appropriate  wave  form  displays. 

A  second  major  challenge  of  the  2-D  simulation  included  momentary  signals  and 
their  associated  holding  circuits.  Most  component  rules  are  in  a  file  that  is 
processed  by  the  program  SIMDEF.  This  program,  as  was  stated  before, 
translates  the  equations  into  a  form  suitable  for  computation  purposes.  Sane 


rules ,  however ,  could  not  be  handled  in  this  manner,  either  because  of  tine 
dependencies  or  because  of  the  complexity  of  the  equation.  Momentary  action 
switches  and  their  associated  holding  circuits  and  three-wire  to  one-wire 
synchro  calculations  had  to  be  handled  in  a  different  fashion.  In  these  cases 
the  conponent  rules  were  written  as  subroutines  in  the  2-D  simulation  program. 

Circuit  values  are  calculated  initially  during  the  initialization  process  for 
the  2-0  simulation  problem.  They  are  subsequently  recalculated  whenever  a 
switch  setting  is  changed  by  the  user  or  whenever  potentiometer  or  handwheel 
controls  are  changed  in  their  position.  Circuit  values  are  also  recalculated 
when  a  component  is  repaired  or  replaced.  During  recalculation  of  circuit 
values,  multiple  passes  through  the  rule  set  are  required  which  continue  until 
all  circuit  values  stabilize  or  until  a  maximum  of  10  cycles  has  been  reached. 
Sane  broken  components  cause  the  values  of  other  circuit  components  to  change 
continually,  and  some  circuit  values  never  stabilize.  For  instance,  if  a  radar 
does  not  lock  on  the  target  which  is  a  symptom  of  many  simulation  problems,  the 
radar  continues  to  search  for  the  target.  This  means  that  a  set  of  several 
synchro  circuits  which  monitor  radar  azimuth  and  elevation  will  be  in  a 
continually  changing  state. 

A  special  provision  during  the  simulation  for  a  HELP  function  was  made  to 
enable  students  to  review  information  or  data  related  to  their  solution  of  the 
problem  before  the  problem  is  terminated.  The  HELP  key  may  be  pressed  at  any 
point  in  the  problem  that  does  not  require  a  specific  input.  Upon  pressing 
HELP  the  student  receives  the  option  to  obtain  information  on  control  key 
functions,  see  a  restatement  of  the  original  problem  as  it  was  presented,  look 
at  a  current  score  report  for  the  problem,  or  quit  and  save  the  problem  for 
restart  at  a  later  time. 

3.2.3  Lesson  Surface  Design 

The  development  of  lessons  for  the  I-HfiWK  DIS  consisted  of  two  main  activities. 
First,  the  development  of  programs  and  videodisc  materials  for  new  lessons; 
and  second,  the  revision  of  existing  lessons  created  during  previous 
development  phases  for  use  in  Phase  III  on  the  DIS. 

During  Phase  III,  twenty-four  new  instructional  lessons  were  developed  (plus 
three  introduction  lessons).  The  original  contract  plan  specified  development 
of  twelve  lessons,  six  of  which  would  be  demonstration  lessons  showing  ISCs 
being  performed,  and  six  of  which  would  be  "backpanel"  lessons  demonstrating 
the  flow  of  signals  and  data  during  the  ISC  Checks.  It  was  originally  felt 
that  this  combination  of  lessons  would  provide  an  adequate  groundwork  of 
information  for  the  solving  of  troubleshooting  problems.  During  the  course  of 
development,  it  became  apparent  that  the  six  signal  flow  lessons  would 
constitute  a  heavy  information  burden  on  the  student,  requiring  more  effort  on 
the  student's  part  than  was  deemed  appropriate  by  USAADS  and  WICAT  personnel. 

Based  on  this  realization,  the  authoring  of  signal  flow  lessons  was  diverted  to 
a  set  of  sixteen  troubleshooting  exercise  lessons  intended  to  serve  as  highly- 
guided  and  path-constrained  exercises  in  troubleshooting.  To  accompany  these 
lessons  and  build  a  foundation  for  their  use,  it  was  determined  that  a  lesson 


would  also  be  required  to  provide  students  with  a  standard,  acceptable 
troubleshooting  process  for  use  on  both  troubleshooting  exercises  and  2-D 
simulation  problems.  At  the  time  of  contracting  such  a  troubleshooting  process 
was  not  available  from  USAADS  either  as  a  doctrinal  statement  or  as  a  concensus 
product.  Ultimately,  a  process  was  outlined  by  USAADS  SMEs  which  satisfies  the 
training  backgrounds  and  practices  of  the  majority  of  experienced  I-HAWK 
trainers  and  field  practitioners. 

It  was  also  determined  that  the  six  ISC  demonstration  lessons  would  have 
reduced  utility  for  the  student  without  certain  prerequisite  information  about 
the  ISC  being  provided  in  an  ISC  introduction  lesson.  Therefore,  this  lesson 
was  also  prepared  for  use  on  the  I-HAWK  DIS  System.  The  final  accounting  of 
lessons  created  during  Phase  III  development  is  as  follows: 


1  Introduction  to  ISC  Lesson 

7  ISC  Demonstration  Lessons 

1  Troubleshooting  Process  Lesson 

16  Troubleshooting  Exercises 

3  Introduction  Lessons 


28  Total  Lessons 


Seven  demonstration  lessons  were  prepared,  rather  than  the  six  required  in  the 
contract.  The  preparation  of  the  seventh  lesson  stemmed  from  the  results  of 
the  first  of  the  two  project  evaluations  (See  Section  4.1)  in  which  it  was 
found  that  the  format  of  lesson  presentation  (high  audio  content)  and  the 
technical  inaccuracy  of  the  existing  HIPIR/LCHR  Check  lesson  hampered  student 
use  of  the  lessons  and  instructor  confidence  in  it.  The  "engineering"  of  the 
les scais  to  meet  student  requirements  and  information  needs  as  described  above 
were  based  on  the  best  available  advice  from  USAADS  and  the  best  judgment  of 
the  contractor.  Experience  has  shown  during  the  evaluation  of  the  lessons  that 
the  lessons  are  an  important  factor  in  the  success  of  the  system,  and  that 
student  performance  would  be  less  satisfactory  without  this  proper  balance 
between  the  lessons  and  the  simulation  exercises. 

Three  lessons  prepared  during  project  Phase  II  were  "remodelled"  for  use  with 
the  newly  created  lessons.  The  approach  in  the  original  three  lessons 
consisted  of  motion  video  accompanied  by  an  audio  message.  These  lessons  were 
remodelled  to  fit  the  general  format  of  the  newly-created  lessons. 

Overall  lesson  organization  into  units  is  shown  in  Table  2  on  page  20.  The 
pattern  of  usage  for  each  lesson  has  already  been  shown  (See  Figure  3  on  page 
9) .  Lessons  may  be  taken  in  one  of  two  modes  —  Instruction  mode,  or  Review 
mode. 
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Table  2 


Lessons  and  Units 
Developed 


Unit  1:  Introduction  to  the  ISC 

Lesson  1:  Introduction  to  the  ISC 


Unit  2:  ISC  Demonstrations 

Lesson  1:  Initialization  Check 
Lesson  2:  Par  Check 
Lesson  3:  Cwar  Check 
Lesson  4:  AADCP  Check 
Lesson  5:  IFF  Check 
Lesson  6:  HIPIR/CCHR  Check 
Lesson  7:  ROR  Check 

Unit  3:  Troubleshooting  Aids 

Lesson  1:  Wire  Tracing  Diagrams 
Lesson  2:  System  Functional  Diagrams 
Lesson  3:  Fault  Isolation  Directory 

Unit  4:  Troubleshooting  Exercises 

Lesson  1:  Introduction  to  Troubleshooting 
Lessons  2-17:  Troubleshooting  Exercises 


Instruction  mode  moves  the  user  through  all  parts  of  the  lesson  to  its 
conclusion.  A  user  selecting  to  take  the  lesson  in  review  mode  is  presented 
with  a  menu  of  the  major  information  blocks  in  the  lesson.  The  blocks  nay  be 
chosen  in  any  order. 

Lesson  contents  are  organized  in  a  highly  modular  and  regular  fashion  to 
facilitate  information  access.  Each  item  of  infomation  within  a  lesson  is 
clearly  labelled  by  type,  and  lesson  structure  for  the  presentation  of 
infomation  consists  of  sequences  of  labelled  blocks  of  information  arranged  in 
a  consistent  pattern.  This  was  found  to  facilitate  initial  instruction  as  well 
as  review  for  students ,  and  lessons  thus  organized  can  be  used  for  either. 

If  a  student  elects  to  take  a  lesson  exam,  the  exam  driver  program  is  invoked. 
The  exam  process  begins  as  an  instructor  enters  an  exam  pass  code.  The  exam 
pass  code  secures  tests  from  student  exploration  and  is  a  safeguard  to  insure 
that  the  test-taker  is  actually  the  student  represented,  preventing  the 
possibility  of  stand-ins.  Following  the  entry  of  the  pass  code,  the  student 
views  a  series  of  computer-generated  test  items  in  multiple  choice  format. 

Test  length  is  controlled  by  an  instructor-set  parameter  for  each  test.  Test 
items  are  sampled  from  a  pool  of  50  (or  less)  randomly  selected  items  and  are 
presented  to  the  student  up  to  3  times  for  reconsideration  of  an  answer  upon 
his/her  request.  Should  a  student  take  too  long  on  a  test,  the  test  is 
automatically  terminated  at  a  given  time-out  point.  This  time  out  point  is 
also  set  by  the  instructors.  When  the  student  has  seen  each  test  item,  the 
exam  routine  asks  if  the  test  has  been  completed.  If  not,  the  routine  recycles 
back  to  the  first  test  item. 

When  the  student  signals  completion  of  the  exam,  the  test  is  scored  immediately 
and  reported  to  the  student  as  a  percentage  correct.  Then  the  exam  routine 
reviews  each  item  in  turn,  the  answer  given,  and  the  correct  answer.  Exam 
items  are  presented  on  subsequent  attempts  based  on  the  student's  response  on 
previous  attempts.  If  a  student  challenges  an  exam  for  a  second  time,  it  is 
possible  to  receive  questions  from  a  previous  test,  but  only  those  which  were 
answered  incorrectly.  Questions  answered  correctly  on  previous  tests  are 
removed  from  the  test  pool  for  that  student. 

TWo  main  interaction  patterns  were  used  in  lesson  design,  as  shown  in  Figure  6 
on  page  22.  The  Type  1  structure  consists  of  the  presentation  of  information 
frames  followed  by  a  review  or  inference  question,  followed  by  feedback. 
Feedback  consists  of  knowledge  of  results  (right  or  wrong)  and  correct  answer 
feedback  for  wrong  responses.  Type  2  lesson  structure  is  used  primarily  in 
troubleshooting  exercises.  It  consists  of  a  frame  containing  an  informational 
message  and  a  multiple  choice  question.  On  an  incorrect  response  the  student 
is  branched  to  a  wrong  answer  feedback  frame,  which  then  returns  the  student  to 
the  original  question  frame.  Correct  response  causes  a  branch  to  a  second 
choice  frame  like  the  first,  with  its  own  informational  message,  question,  and 
options. 


Figure  6 


Lesson  Interaction 
Structures 


3.2.4  Lesson  Internal  Design 

The  creation  and  use  of  lessons  is  through  "builder"  and  "driver"  programs. 

This  eliminates  the  need  for  individual  programs  for  each  lesson.  Instead ,  the 
driver  program  stores  lesson  data  in  a  file  (shown  in  Figure  7  on  page  24) . 

When  acted  upon  by  the  lesson  driver  program,  the  data  file  executes  the 
lesson.  It  was  possible  using  the  builder  and  driver  programs  to  create  all 
I-HAWK  lessons.  This  approach  to  lesson  creation  simplified  data  entry, 
reduced  programmer  requirements,  and  siirplified  debugging  and  testing  of 
lessons.  In  the  future,  the  use  of  these  programs  will  simplify  revision  of 
the  instructional  materials  by  the  training  developer. 

3.2.5  Utility  Program  Design 

Four  programs  were  created  for  utility  purposes  to  help  the  I-HAWK  DIS  complete 
administrative  as  well  as  instructional  functions:  the  registration  program, 
the  report  generator  program,  the  simulation  problem  generator,  and  the  exam 
maintenance  program. 

The  registration  program  enrolls  students,  instructors  and  managers  on  the 
I-HAWK  DIS  and  assigns  each  a  user  number.  The  registration  program  creates  a 
record  file  for  each  enrolled  user  in  which  the  data  used  by  the  report 
generator  program  is  sorted.  The  registration  program  also  has  facilities  for 
changing  user  data  including  the  user  number,  name,  and  category,  and 
capabilities  for  deleting  registered  users. 

The  report  generator  program  acts  upon  lesson  test  and  simulation  problem  score 
data  and  creates  a  series  of  reports  on  student  performance,  individually  and 
by  group,  and  on  the  performance  of  lesson  exam  items  and  simulation  problems. 
The  data  in  these  reports  may  be  used  to  identify  faulty  exam  items  and  weak 
areas  of  instruction,  to  grade  the  difficulty  of  simulation  problems,  or  to 
indicate  remediation  for  individual  students. 

The  2-D  simulation  problem  generator  allows  the  instructor  to  create  a 
simulation  problem  and  assign  it  to  one  or  more  students.  This  program  also 
allows  instructors  to  declare  problems  completed  by  a  student.  Through  careful 
use  of  the  simulation  problem  generator,  course  developers  may  create  a 
carefully  graded  sequence  of  simulation  problems — from  relatively  simple 
through  relatively  sophisticated  problems — in  an  organized  fashion  to  the 
students. 

The  exam  maintenance  program  allows  developers  to  control  the  content,  length, 
scoring  criteria,  and  time-out  point  of  tests.  The  program  is  accessed  through 
a  workstation  by  developers  so  that  they  may  see  how  test  items  appear  on  the 
student  terminal  screen.  Using  the  exam  maintenance  program,  developers  may 
create,  modify  or  delete  items  from  the  item  pool.  In  addition,  they  may 
modify  the  test  parameters  for  each  exam,  including  specifying  the  number  of 
items  to  be  presented,  the  length  of  the  time  out  period,  and  the  criterion  for 
a  passing  score.  The  test  maintenance  program  has  been  designed  for  easy  use 
and  includes  a  screen  editing  capability  that  is  easy  to  use. 


■■m 


* 

& 


LESSON  DATA 
FILE 


Figure  7 

Lesson  Programming 
Process 


*  ^,1 


4.0  Project  Organization 

This  section  describes  the  work  plan  used  for  Phase  III  activities  and  names 
the  organizational  responsibilities  for  the  program. 

4.1  Tasks  and  Schedule 

The  work  completed  during  system  development  was  divided  into  five  tasks, 
accomplished  according  to  the  Contract  Performance  Plan  in  Figure  8  on  page  26. 


Task  1  consisted  of  system  installation  and  support  functions  and  took  place  in 
three  subtasks. 

The  equipment  described  in  the  previous  section  was  installed  during  Subtask 
1.1  at  the  I-HAWK  Division,  USAADS,  at  Ft.  Bliss,  Texas.  The  standard  power 
requirements  of  the  system  facilitated  this  process.  No  special  provisions 
were  required  for  environmental  control  except  for  the  shielding  and  grounding 
of  critical  system  components  to  avert  the  effects  of  local  radar  emanations. 

Once  the  system  was  installed.  Subtask  1.2  required  the  training  of  Army 
personnel  in  the  operation  and  operational  maintenance  of  the  system. 

Subtask  1.3  called  for  contractor  maintenance  support  for  the  system  on  an  "as 
required"  basis.  This  support  included  replacement  of  faulty  components,  and 
updates  of  system  programs  as  required. 


Task  2  provided  for  an  evaluation  to  demonstrate  that  the  system  hardware  and 
operating  software  were  capable  of  supporting  instructional  functions  reliably. 
The  programs  and  equipment  were  to  be  tested  for  accurate  operation  and 
endurance.  A  set  of  benchmark  expectancies  were  formulated  prior  to  the 
evaluation  as  criteria  for  these  tests.  Subtask  2.1  consisted  of  the 
evaluation  itself.  Subtask  2.2  consisted  of  changes  to  hardware  and  software 
based  on  the  evaluation  findings. 

Part  of  the  Task  2  evaluation  was  also  to  focus  on  the  determination  of  which 
existing  lessons  developed  under  the  previous  contract  should  be  retained. 


Under  Task  3,  additional  instructional  materials  were  to  be  developed.  A  set 
of  lessons  specified  in  the  contract  were  to  be  created  for  computer-controlled 
videodisc  presentation.  These  were  to  be  supplemented  by  lessons  retained  from 
the  previous  contract  phase. 

Subtask  3.2  involved  the  development  of  lessens,  beginning  with  the  design  of 
lesson  formats  specification  and  policies  and  continuing  through  content 
gathering,  authoring,  and  review  of  finished  lessons.  A  total  of  12  lessons 
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were  required  by  the  contract,  half  of  them  demonstrating  1-HAWK  test 
procedures,  and  half  dealing  with  I-HAWK  equipment  signal  flews  during  those 
test  procedures. 


Following  development  of  the  lessons,  which  was  accomplished  first  in  script 
form.  Subtask  3.2  required  the  production  of  lessons  on  videodisc.  This 
process  required  the  shooting  of  motion  and  still  video  material  using  I-HAWK 
equipment  at  USAADS,  Ft.  Bliss.  Footage  shot  was  then  catalogued  and  prepared 
for  studio  editing,  as  were  the  character-generated  text  material.  At  the 
studio,  an  edited  master  videotape  was  prepared.  This  master  was  sent  to  a 
production  agency  for  production  of  videodiscs. 

Sub task  3.3  specified  a  programming  effort  to  be  carried  out  parallel  to 
videodisc  production.  Programs  to  drive  videodisc  lesson  presentation  were 
prepared. 

During  Sub task  3.4  the  disc  and  program  material  was  installed  at  Ft.  Bliss  in 
preparation  for  evaluation. 

Task  jl  2-n  SiimilaHnn 

As  Task  3  called  for  preparation  of  lesson  material.  Task  4  required  the 
preparation  of  a  2-0  simulation  administered  by  computer-controlled  videodisc. 
The  simulation  was  to  pertain  to  the  circuits  used  for  Step  1  of  the  HIPIR/LCHR 
ISC  Check  for  the  I-HAWK  System. 

The  product  of  Subtask  4.1  was  a  specification  of  problems  to  be  administered 
by  the  2D  simulation,  including  a  description  of  the  program  requirements. 

From  that  specification.  Subtask  4.2  generated  the  problems  themselves, 
including  detailed  program  specifications  and  scripts  for  videodisc  production. 

Subtask  4.3,  2D  simulation  disc  production,  was  a  process  identical  to  Subtask 
3.2  for  CAI  videodisc  production.  It  was  anticipated  that  footage  shot  for 
instruction  would  be  usable  for  simulation  also. 

Concurrently,  programming  for  the  simulation  occurred  during  Subtask  4.4, 
beginning  as  soon  as  specifications  were  complete. 

Following  programming,  Subtask  4.5  involved  the  debugging  and  testing  of  the 
programs  prior  to  installation  and  evaluation. 

TflfiK  5i  Total  Syitm  Evaluation 

During  the  Task  5  evaluation  the  functionality,  effectiveness,  and  productivity 
of  the  system  and  its  programs  and  materials  were  measured.  Reports  written 
during  prior  project  phases  were  to  be  used  as  process  and  criterion  reference. 

Sub task  5.1  provided  for  data  gathering  activities,  and  Subtask  5.2  provided 
for  the  revision  of  programs  and  presentations  based  on  evaluation  outcomes. 
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4.2  Organizational  Participation 

The  accomplishment  of  the  tasks  described  in  the  previous  section  was  through  a 
joint  cooperative  effort  between  TDI,  ACT 0,  USAADS,  and  WICAT,  as  stated  in 
Section  1.0.  Each  of  these  organizations  accepted  a  role  in  the  development 
effort  as  shown  in  Figure  9  on  page  29. 


5.0  Results  and  Outcomes 

This  section  describes  outcomes  of  performing  the  tasks  described  in  Faction 
4.0  and  the  results  obtained  using  the  DIS  system  and  materials. 

By  way  of  caution/  it  should  be  emphasized  that  this  project  was  not  a  research 
project.  Its  aim  was  the  testing  of  a  concept  for  electronic  maintenance 
training  using  state-of-the-art  training  technology.  The  evaluations  conducted 
during  contract  task  5  produced  student  performance  data  collected  in  a 
realistic  environment  not  subject  to  controls  normally  used  in  research  and 
full  of  the  inconveniences  of  the  real  training  world.  The  data  reported  must 
not  be  treated  as  research  data,  but  rather  as  evaluation  or  action  research 
data.  Statistical  analysis  has  been  sparing  to  avoid  the  implication  that  it 
can  be  so  treated. 

5.1  Task  1 

The  installation  of  the  DIS  equipment  took  place  early  in  the  contract  period. 
The  installation  originally  included  workstations  operating  directly  from  a 
floppy  disk  to  provide  the  eleven  lessons  of  instruction  created  during  Phase 
II.  Installation  of  additional  workstations  and  storage  station  hardware  took 
place  by  March,  1981,  followed  four  months  later  by  the  installation  of  the 
networking  software  and  hardware  modifications  to  support  the  network  programs. 
Throughout  the  project,  several  problems  were  identified  in  individual  system 
components  including  central  processing  unit  boards,  memory  boards,  keyboards 
and  terminals .  As  problems  were  identified,  the  appropriate  replacements  or 
repairs  were  made. 

Training  of  Amy  personnel  in  system  operation  and  operational  maintenance  took 
place  on  a  continuing  basis  throughout  the  contract  period,  but  more 
particularly  during  the  final  stages  of  program  development.  In  addition  to 
direct  training  on  system  operation,  many  modifications  to  control  programs  and 
system  booting  procedures  reduced  the  amount  of  operator  involvement  to  a 
minimum  and  simplified  the  operator's  task.  A  s unitary  of  system  operating 
procedures  in  the  latest  version  is  contained  in  Appendix  B  and  a  summary  of 
system  maintenance  procedures  is  presented  in  Appendix  C. 

5.2  Task  2 

Task  2  hardware  and  software  evaluation  was  carried  out  midway  in  the  contract 
period.  The  plan  which  formed  the  basis  for  that  evaluation  is  presented  in 
Appendix  D.  The  results  of  that  evaluation  determined  the  soundness  of 
operation  for  the  system,  its  operating  software  and  its  networking  software. 
The  subjects  of  the  evaluation  for  the  eleven  lessons  created  during  Phase  II 
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Organization 


Function 


Directorate  of  Training 
Developments,  U.S.  Army 

Air  Defense  School  (USAADS) 
Ft.  Bliss,  Texas 

Provided  subject  matter  expertise  and  technical 
material. 

Supplied  current  training  data  and  course 
outlines. 

Supplied  simulation  circuit  data. 

Reviewed  lesson  scripts,  programs  and  simulation 
programs. 

Participated  in  studio  editing  process  as 
technical  reviewer. 

U.S.  Army 

Training  Developments 
Institute  (TDI) 

Ft.  Monroe,  Virginia 

Provided  project  funds. 

Monitored  contract. 

Reviewed  plans  and  proposed  designs. 

Offered  technical  consulting  on  designs. 

Amy  Communicative 

Technology  Office  (ACTO) 

Ft.  Eustis,  Virginia 

Monitored  contract. 

Reviewed  plans  and  proposed  designs. 

Offered  technical  consulting  on  designs. 

WICAT  Systems 

P.  0.  Box  539 

1875  South  State  Street 

Orem,  Utah 

Carried  out  tasks  and  schedules  for  design  and 
development  of  the  system  as  described. 

were  run  according  to  the  schedule  described  in  the  evaluation  plan  and 
measured  against  the  criteria  stated  there. 

The  evaluation  demonstrated  the  general  soundness  of  the  system.  In  addition, 
behavior  of  system  software  and  hardware  resulted  in  several  modifications  and 
improvements.  Improvements  included  the  decision  to  use  the  hand-held  keypad 
for  student  interface ,  revision  of  system  operating  procedures  for  ease  of  use 
by  system  operators,  and  the  modification  of  system  networking  programs  for 
improved  reliability  and  speed. 

5.3  Task  3 

During  Task  3,  the  additional  instructional  materials  described  in  Section  3.0 
of  this  report  were  developed,  and  lessons  which  had  been  created  during 
previous  phases  of  the  project  were  modified. 

Lessons  were  authored  using  Amy  technical  documentation  and  the  a* ill  and 
knowledge  of  Amy  subject-matter  experts.  Course  material  was  collected  on  a 
set  of  special  forms  designed  for  information  collection.  These  capitalized 
upon  the  regular  informational  structure  of  the  lessons.  The  forms  were 
completed  from  technical  manual  information  by  contractor  personnel  and  were 
then  submitted  to  Amy  subject-matter  experts  for  completion  and  for  supplying 
of  information  not  available  in  the  technical  manuals.  In  this  fashion  content 
was  gathered  over  long  distances  for  a  large  number  of  lessons  involving  highly 
technical  content.  The  authoring  of  the  troubleshooting  exercise  lessons  was 
accomplished  with  dedicated  in-person  working  sessions  for  lesson  authoring. 
Following  format  design  of  these  lessons,  WICAT  personnel  worked  directly  with 
Army  subject-matter  experts  to  author  them. 

Lessons  existed  from  Phase  II  and  materials  for  the  Glossary  were  examined 
during  this  period  aid  reconditioned  to  fit  format  requirements  for  the  new 
lessons. 

lesson  authoring  took  place  using  a  special  set  of  production  sheets.  The 
sample  production  sheet  is  presented  in  Figure  10  on  page  31.  The  production 
sheet  was  designed  especially  for  the  creation  of  computer  controlled  videodisc 
lessons  and  contains  information  needed  by  a  variety  of  production  personnel 
involved  in  lesson  creation;  they  include  television  studio  personnel  required 
during  editing,  television  technicians  required  during  preparation  of  text 
frames,  photographers  and  video  specialists  required  during  photographing  and 
videotaping  of  required  visual  materials,  graphic  artists  required  during  the 
production  of  required  graphics  materials,  instructional  developers  and 
subject-matter  experts  required  during  the  authoring  and  review  of  lesson 
materials,  and  computer  programmers  required  during  the  creation  of  lesson 
programs.  The  production  sheet  used  on  this  program  was  a  customization  of 
production  sheets  used  on  other  projects. 

A  process  of  reviews  was  set  up  in  conjunction  with  Army  subject-matter  experts 
for  the  purpose  of  insuring  the  technical  accuracy  and  appropriate  level  of 
presentation  of  the  instructional  materials.  These  reviews  were  imbedded  in  a 
set  of  clearly  defined  authoring  procedures  which  covered  the  actions  of  both 
contractor  lesson  writers  and  Army  lesson  reviewers. 
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Instructions:  □ff  Qsf 


The  production  of  materials  leading  to  the  creation  of  the  instructional 
videodiscs  began  with  on-site  visits  at  Ft.  Bliss  for  the  purpose  of 
photographing  and  videotaping  still  pictures  and  motion  sequences  of  I-HftWK 
System  equipment  and  operation.  Existing  materials  photographed  and  videotaped 
Airing  the  Phase  II  development  effort  were  also  catalogued  and  prepared  for 
use  in  the  making  of  Phase  III  videodiscs.  Concurrently/  with  the  shooting  of 
footage  at  Ft.  Bliss/  preparation  began  at  the  contractor's  work  site  for  video 
text  frames.  In  total#  the  production  effort  expended  for  Phase  III  production 
of  lesson  materials  included  a  total  of  6634  individual  character  generator 
text  pages#  1620  individual  slides  and  120  minutes  of  videotape  footage  shot. 

During  studio  editing  of  the  videotape  master#  500  of  the  1600  slides  were 
actually  used  and  a  total  of  7000  edits  were  made. 

The  master  videotape  was  sent  for  videodisc  production  to  DiscoVieion 
Associates#  and  a  check  tape  was  requested  for  proofing  of  the  disc  master 
contents. 

Programming  of  the  lesson  builder  and  lesson  driver  programs  proceeded  in 
parallel  with  the  production  process#  and  upon  completion  of  the  programs  data 
was  altered  for  lessons  and  lessons  were  tested. 

5.4  Task  4 

The  greatest  requirement  for  data  transfer  between  Army  subject-matter  expects 
and  the  contractor  came  Airing  the  creation  of  simulation  programs  and  data 
bases.  Simulation  data  gathering  involved  identification  of  components  to  be 
simulated  and  a  charting  of  the  behavior  of  each  component  through  the 
construction  of  a  rule  set  describing  component  behavior .  Each  rule  expresses 
the  influence  of  the  component  on  signals  input  to  it.  Data  gathering  on  the 
simulation  components  took  place  early  in  the  project  and  continued  until  the 
end  of  the  project  on  an  intensive  basis. 

Photography  and  videotaping  and  generation  of  video  text  pages  for  the 
simulation  was  combined  with  that  for  the  instruction.  Twelve  hundred  such 
pages  were  created  for  the  simulation  alone.  Three  thousand  five  hundred  sixty 
slides  were  taken.  Sixty  minutes  of  videotape  were  shot#  and  ultimately  3200 
slides  were  used  in  the  production  of  the  simulation.  A  total  of  4083  edits 
were  made  for  the  simulation.  An  Army  subject-matter  expert  was  present  for 
the  editing  process  both  for  the  simulation  and  lesson  editing. 

The  evaluation  took  place  in  four  phases: 

•  Phase  0  -  Quality  assurance  and  pre-installation  formative  review 

•  Phase  I  -  Instructor  training  and  orientation  during  installation 

•  Phase  II  -  Evaluation  of  full  system  implementation  with  individual 

and  grouped  students 

•  Phase  III  -  Full  class  system  evaluation 
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5.5.1  Phase  0  Results 


Phase  0  activities  consisted  of  a  formative  review  of  lessons  by  a 
representative  population  of  I-HAWK  Maintenance  Repair  Course  students.  As 
these  were  not  available  to  the  contractor  locally ,  students  from  Utah 
Technical  College  and  Brigham  Young  University  were  used.  Ihe  review  of 
materials  was  integrated  with  data  entry.  After  a  block  of  data  was  entered, 
it  was  reviewed  for  data  accuracy  and  message  values. 

Ihe  results  of  this  review  were  mixed,  and  fell  somewhat  below  expectations. 
Many  changes  to  lessons  did  take  place  as  a  result  of  the  review,  but  not  all 
that  were  required.  Consequently,  the  review  and  correction  begun  at  WICAT's 
work  site  had  to  be  completed  at  the  test  site  prior  to  use  of  the  system  by 
students.  By  the  tine  Amy  students  interacted  with  the  lessons,  most  major 
problems  had  been  solved,  and  any  additional  problems  encountered  were 
relatively  minor  and  easily  corrected.  The  efforts  of  the  Subject-Matter 
Experts  (SME)  during  this  test  site  review  were  invaluable.  Both  content  and 
data  errors  not  detected  earlier  were  spotted  and  corrected  during  the  review. 

Following  installation  of  programs  at  the  Ft.  Bliss  test  site,  tryout  of  the 
materials  with  student-like  subjects  was  possible,  using  one  24C10  student  and 
one  Warrant  Officer  who  thoroughly  reviewed  the  lessons.  Following  their 
review,  each  was  given  a  questionnaire  to  complete  and  was  interviewed.  The 
result  of  this  data  gathering  is  summarized  below.  The  MQS  24C10  student  found: 

1.  The  regular  pattern  of  lesson  organization  was  very  agreeable,  since  it 
allowed  him  to  leave  a  lesson  and  return  to  it  later  without  the  need 
to  "warm-up"  again. 

2.  The  embedded  quizzes  were  very  useful  for  information  recall.  The 
student  recotnnended  the  creation  of  a  carry-away  data  book,  a 
suggestion  not  implemented  due  to  the  possibility  it  might  dilute 
dependence  on  the  Technical  Manuals. 

3.  The  videodisc-based  instruction  was  deemed  superior  to  book 
instruction.  The  student  expressed  an  interest  in  further  instruction 
using  this  medium.  (The  student's  interest  was  demonstrated  as  he 
returned  twice  on  his  own  free  time  to  view  lessons.)  This  opinion 
persisted  even  when  at  one  point  seme  program  difficulties  were  being 
experienced. 

4.  The  student  recommended  the  inclusion  of  more  audio,  though  he  found 
the  text  easy  and  clear  to  read.  He  said  he  often  became  bored  with 
other  mediated  lessons  but  found  these  lessons  interesting. 

5.  The  student  enphasized  the  importance  of  the  instruction  as  a  question- 
answering  resource. 

The  Warrant  Officer,  an  experienced  user  of  the  ISC,  indicated  the  following: 

1.  The  lessons  cleared  up  many  details  which  were  previously  unclear  and 
provided  a  valuable  new  perspective.  Some  of  the  Troubleshooting  Aid 
lessons  clarified  information  which  he  had  misunderstood  before. 


33 


2.  Use  of  the  lessons  within  HAWK  batteries  was  recoimended  because  they 
contain  information  on  procedures  and  Technical  Manual  data  unfamiliar 
to  most  field  personnel. 

3.  The  textual  approach  was  preferred  to  the  audio,  but  the  delay  in 
loading  a  lesson  onto  the  work  station  from  the  Storage  Station  was 
considered  tedious.  This  remark  was  made  almost  universally  by 
evaluation  subjects. 

4.  Hie  hand-held  keypad  was  preferred  over  the  keyboard  for  inputting 
responses . 

5.  The  carrel  work  surface  was  felt  to  be  too  small  and  needed  a  table  to 
be  placed  in  front  of  it.  This  opinion  was  also  widely  held  among 
evaluation  subjects. 

5.5.2  Phase  I  Results 

Training  of  instructors  took  place  at  first  in  early  March,  1982  with  a  group 
of  three  instructors.  Adjustments  in  the  assignments  of  the  instructors  made 
it  necessary  to  train  a  second  group  of  two  instructors  later  during  the  month 
of  March.  Instructor  training  ran  two  days  and  consisted  of  a  brief 
introduction  and  demonstration  of  the  equipment,  the  instructional  and 
simulation  materials  which  had  been  prepared,  and  the  procedures  for  operating 
the  system  to  implement  the  materials.  Following  this,  instructors  were 
allowed  to  take  lessons  and  exams  as  students.  User  aids  which  were  available 
at  training  time  were  demonstrated  to  the  instructors.  As  new  materials 
became  available,  they  also  were  demonstrated  for  the  instructors. 

The  changing  role  of  the  instructor  during  the  evaluation  was  stressed 
strongly,  as  it  was  seem  as  a  new  role  and  different  from  the  one  they  were 
used  to  assuming.  The  instructors  were  briefed  on  the  goals  and  procedures  of 
the  evaluation  and  their  role  in  carrying  out  the  evaluation.  That  role  was 
portrayed  to  them  as  an  opportunity  to  direct,  answer  questions,  probe  for 
student  problems,  diagnose,  and  remediate,  rather  than  to  act  as  the  primary 
source  of  information.  These  new  functions  were  ernes  they  themselves  had 
stated  were  more  important  than  information  delivery.  The  goal  was  to  show 
their  time  was  being  reallocated  to  functions  they  knew  were  more  important 
and  more  difficult.  It  is  felt  that  this  perspective  was  critical  to  the 
success  of  the  evaluation  and  that  in  future  applications  of  the  system  it  will 
be  an  important  key  to  success. 

Following  the  instructor  training,  the  two  instructors  were  interviewed  and 
given  questionnaires  to  complete.  The  results  were  very  favorable,  and  a 
summary  of  the  comments  and  questionnaire  results  is  given  below: 

1.  A  small  number  of  inconsistencies  in  the  lesson  content  and  ISC 
procedure  were  noted. 

2.  Several  details,  pertaining  to  I -HAWK  training  practices  were 
recommended  for  inclusion  in  the  lessons,  to  help  students  identify 
deviation  from  Technical  Manual  documentation. 


3.  Both  instructors  remarked  that  they  were  sure  students  would  appreciate 
the  lessons  and  offered  advice  on  methods  for  using  the  lessons  with 
students  individually  and  in  groups.  One  instructor  did  use  the 
lessons  during  training  with  a  group  of  Allied  students  and  said  the 
response  of  the  students  was  very  positive. 

4.  Both  instructors  suggested  courses  other  than  the  MDS  24E10  course 
which  could  benefit  from  the  use  of  the  lessons.  Recoimendations  for 
tailoring  the  lessons  to  those  audiences  were  also  given,  including 
adjusting,  in  seme  cases,  the  level  of  detail  given  in  a  lesson.  The 
level  of  content  was,  overall,  seen  as  appropriate  for  MDS  24E10. 

5.  One  instructor  felt  that  the  other  instructors  should  take  the  lessons 
and  felt  that  this  would  convince  them  of  the  soundness  of  the  method 
of  instruction. 

6.  One  instructor  recommended  the  use  of  the  system  at  a  tactical  site  as 
a  training  tool  for  skill  maintenance. 

7.  Both  instructors  spontaneously  volunteered  to  return  at  the  next 
opportunity  to  lode  at  additional  lessons  and  did  so. 

Based  on  the  responses  of  the  instructors  to  this  training,  the  materials  were 
modified  and  enhanced,  and  improvements  to  the  Lesson  Driver  program  were 
implemented.  In  addition,  slight  changes  in  the  strategy  used  with  students 
were  considered. 

5.5.3  Phase  II  Results 

Formal  evaluation  was  conducted  using  three  separate  evaluation  groups  of  MDS 
24E10  students.  A  total  of  46  students  were  treated  for  data  gathering 
purposes.  Sixteen  students  made  up  a  control  group  who  were  given  normal 
classroom  instruction  followed  by  a  performance  post-test  administered  by 
evaluation  personnel.  The  remaining  30  students  were  given  sane  combination  of 
CAI  lessons  and  2-D  simulation  problems  interacting  with  the  DIS,  followed  by 
the  same  performance  post-test.  Of  the  30  students  who  were  instructed  on  the 
DIS,  10  were  given  CAI  lessons  only.  The  remaining  20  students  were  given  both 
CAI  lessons  and  2-D  simulation  problems.  The  split  selection  of  students  from 
three  successive  classes  (classes  1,  2,  and  3)  of  MDS  24E10  students  caused  a 
variation  in  the  sample  sizes  between  the  control  group  and  the  two  evaluation 
groups. 

Randan  selection  of  students  within  each  class  to  the  three  evaluation  groups 
was  not  possible.  Selection  was  conducted  by  class  instructors  who  were 
advised  by  the  evaluators  to  select  randomly,  but  the  instructors  did  not 
report  their  methods  of  selection  and  were  not  willing  for  selection  to  be  made 
by  the  evaluator  using  formal  tools.  Evaluators  needs  and  interests  were 
accommodated  in  sane  cases  by  instructors  as  requested,  but  not  in  all.  The 
resulting  class  and  total  distribution  of  students  among  the  three  classes  is 
shown  in  Figure  11. 
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Distribution  of  Students  Across  Groups 


5. 5. 3.1  Procedures  for  Phase  II 

A  standard  method  of  treating  students  was  developed  for  the  evaluation. 
Students  were  greeted  by  an  instructor  who  briefed  them  on  the  purpose  of  the 
instructional  procedure.  Students  were  then  assigned  a  workstation  and  student 
number  for  log-on  purposes.  Each  student  was  then  given  a  lesson  assignment 
record  and  the  instructor  described  the  agenda  of  lessons  and/or  simulation 
problems  for  the  day.  Students  were  then  instructed  to  enter  their  student 
number.  Instructional  sequences  were  presented  and  guided  the  student  through 
instructional  presentations  on  using  the  keypad  keys  to  control  the  system. 
Breaks  were  inserted  during  the  instructional  period  at  spaces  approximately  an 
hour  apart,  and  all  students  were  notified  of  the  break  and  allowed  to  go  at 
the  same  time.  At  the  end  of  the  instruction  period,  or  as  soon  as  the  student 
completed  the  agenda  of  lessons  and  problems  assigned,  they  were  presented 
closing  remarks  and  instructions  for  the  succeeding  instructional  or  testing 
period  by  the  instructor. 

In  order  to  obtain  data  on  the  various  methods  of  presenting  instructional 
materials  and  simulation  problems,  the  daily  routine  of  the  evaluation  was 
changed  between  evaluation  Classes  1  and  2.  Class  1  instruction  was  provided 
in  an  initial  one  hour  instructional  period  and  a  second  eight  hour 
instructional  period.  It  occurred  to  the  evaluators  that  the  unbroken  eight 
hour  period  of  system  use  might  introduce  a  fatigue  factor  for  students.  It 
was  also  found  that  nine  hours  of  instruction  rushed  the  students  too  much.  It 
was  decided  with  Class  2  to  increase  the  total  instructional  time  from  nine 
hours  to  twelve  hours  and  to  provide  those  twelve  hours  as  three  blocks  of  four 
hours  each. 
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Students  in  all  classes  received  roughly  the  same  body  of  CAI  instruction 
consisting  of: 

-  Introduction  to  ISC's  Lesson 

-  Initialization  Check  Demonstration  Lesson 

-  PAR  Check  Demonstration  Lesson 

-  Step  1  of  the  HIPIR/LCHR  Demonstration  Lesson 

-  Fault  Isolation  Directory  Lesson 

-  Introduction  to  Troubleshooting  Lesson 

-  Troubleshooting  Exercises  (1-3) 

-  Simulation  Problems  (1-3) 

Class  1  did  not  complete  troubleshooting  exercises  but  was  able  to  take  the 
other  lessons  listed  above.  All  students  in  Classes  2  and  3  completed 
troubleshooting  exercises.  Students  were  instructed  to  work  at  their  own  speed 
insofar  as  possible.  In  roughly  half  the  cases,  two  students  were  assigned  to 
work  cm  one  terminal  together.  Because  students  worked  at  their  own  speed,  no 
control  was  attained  over  the  exact  amount  of  material  covered.  Amounts 
covered  varied  widely  due  to  severed,  factors,  including  sickness,  pay  check 
pickup,  and  barracks  duty,  all  of  which  drew  students  away  from  all  evaluation 
groups  in  an  unpredictable  pattern. 

No  pretest  was  used  in  any  of  the  groups  to  assess  incoming  ability,  skill,  or 
knowledge  level. 

Because  of  time  constraints  during  the  testing,  the  test  item  administered 
consisted  of  one  fault  isolation  test,  with  the  same  problem  being  used  for  all 
students.  During  the  examination  the  student  would  interface  with  a  subject- 
matter  expert  and  a  data  recorder.  The  subject-matter  expert  would  present  the 
original  problem  (see  Appendix  E  for  the  problem  presented)  and  respond  to 
questions  by  the  examinee.  The  examinee  could  ask  for  operations  to  be 
performed  such  as  setting  a  control,  running  a  test,  or  viewing  an  indicator. 
Students  asking  to  take  measurements  were  asked  for  probe  placement  points  and 
type  of  measurement  to  be  made  (e.g.,  DC  voltage) .  Responses  of  the  tester 
were  limited  to  responses  to  student-initiated  inquiry,  and  no  information 
related  to  the  problem  was  volunteered. 

The  recorder  wrote  the  words  of  the  instructor  and  the  student  during  the  test. 
The  recorder  also  timed  student  problem  solving  time  using  a  stopwatch 
beginning  with  the  completion  of  the  statement  of  symptoms  by  the  tester  and 
ending  when  a  correct  solution  was  achieved.  The  problem  was  allowed  to 
proceed  until  the  student  admitted  verbally  that  his/her  resources  were 
exhausted  or  that  he/she  had  no  idea  what  to  do  next.  In  a  very  few  cases, 
when  the  time  elapsed  had  exceeded  20  minutes  and  no  solution  had  been  reached 
and  the  student's  actions  indicated  aimlessness  or  randomness,  it  was  asked 
whether  the  student  had  an  idea  of  what  to  do  and  whether  the  student  chose  to 
terminate  the  problem.  In  no  case  was  the  termination  forced  if  the  student 
was  willing  to  proceed.  The  tester  always  indicated  a  willingness  to  proceed 
with  the  test  if  the  student  desired. 

The  period  between  instruction  and  examination  was  from  one  to  three  days.  The 
length  of  the  period  appeared  to  have  no  effect  on  the  performance  of  the 

students. 


5.5.3. 2  Data  Sunrary 

A  sunnary  of  the  test  results  is  presented  in  Figures  12  and  13  on  pages  39  and 
40.  Figure  12  presents  a  sunnary  of  student  problem  solving  times  for  each 
group.  Figure  13  sunnarizes  the  number  of  students  in  each  group  completing 
the  terminal  fault  isolation  test  problem  successfully.  The  original  plan  was 
to  devise  an  elaborate  scheme  related  to  the  problem  solving  procedure  which 
would  give  credit  for  each  step  of  the  troubleshooting  procedure  properly 
applied,  but  in  fact  the  large  number  of  students  in  the  control  group  who  did 
not  even  complete  the  problem  before  it  was  terminated  indicated  that  a 
detailed  level  of  scoring  was  unnecessary. 

The  following  conclusions  are  drawn  from  Figure  12  and  13  data.  First,  only  25 
percent  of  the  students  in  the  control  group  were  able  to  solve  the  test 
problem.  Second,  since  the  instruction  only  evaluation  group  was  successful 
without  exception  in  solving  the  problem,  we  conclude  that  the  CAI  instruction 
contains  sufficient  information  required  by  the  students  to  solve  integrated 
system  troubleshooting  problems  within  the  domain  tested.  Third,  since  the 
instruction  and  simulation  evaluation  group  students  solved  the  problem 
correctly  and  did  it  in  half  the  average  time  of  the  instruction  only  group,  we 
expect  that  the  simulation  has  the  effect  of  further  inproving  the  students' 
efficiency  at  problem  solving  once  that  skill  has  been  established.  This  is 
not  clearly  supported,  however,  due  to  the  confounding  of  two  other 
factors — lack  of  troubleshooting  exercise  experience  and  changed  instructional 
day  format — both  of  which  were  true  of  CAI  only  students  and  not  of  CAI  and 
simulation  students. 

No  inference  is  made  concerning  the  breadth  of  the  skill  which  is  established 
by  the  instruction  or  the  simulation.  Additional  evaluations  to  determine  this 
are  required  and  recomnended.  Preliminary  data  gathered  during  the  evaluation 
through  the  testing  of  seme  students  on  more  than  one  problem  indicates  that 
for  evaluation  group  students  receiving  instruction  only,  the  effect  is 
consistent  across  all  types  of  circuit  covered  in  the  instruction  and 
troubleshooting  exercises  and  would  thus  be  expected  to  generalize  across  all 
circuits  which  the  student  has  been  exposed  to  during  training  on  the  DIS. 

Attitudinal  questionnaires  were  filled  out  by  students  to  determine  the 
relative  preference  of  students  for  the  instruction  they  had  received  and  to 
identify  what  they  considered  to  be  its  strengths  and  weaknesses.  Of  the  46 
questionnaires  handed  out  for  completion,  only  21  were  returned,  as  the 
collection  was  done  by  instructors.  A  sample  questionnaire  is  presented  in 
Appendix  F. 

General  trends  appeared  within  the  questionnaire  data.  The  control  group 
preferred  the  instructional  method  used  in  the  DIS  over  other  training  methods. 
Especially  notable  in  this  preference  was  the  high  degree  of  positive  feeling 
toward  the  troubleshooting  exercise  lessons.  Only  one  form  of  training 
received  a  higher  favorable  response  among  students  and  that  was  the  practical 
exercise  using  real  equipment.  It  is  apparent  that  even  though  students  reach 
the  level  of  skilled  performance  using  lessons  and  simulation,  they  do  not  feel 
fully  confident  of  their  skills  until  they've  had  the  opportunity  of  trying 
them  on  real  equipment.  This  is  reasonable,  since  it  is  real  equipment  which 
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Control  Group 


Evaluation  Groups 


_ 

No  CAI  Instruction 

No  Simulation 

CAI  Instruction  Only 

No  Simulation 

CAI  Instruction  and 
Simulation 

(n  -  16) 

(n  =  10) 

(n  =  20) 

12  min.  45  sec.  (T) 

4  min.  5  sec. 

2  min.  0  sec. 

23  min.  44  sec.  (T) 

7  min.  14  sec. 

13  min.  55  sec. 

24  min.  4  sec.  (T) 

8  min.  0  sec. 

14  min.  0  sec.  (T) 

5  min.  45  sec. 

17  min.  52  sec.  (T) 

8  min.  50  sec. 

5  min.  12  sec. 

18  min.  19  sec.  (T) 

11  min.  55  sec. 

4  min.  7  sec. 

11  min.  8  sec. 

14  min.  55  sec. 

4  min.  3  sec. 

18  min.  39  sec. 

14  min.  20  sec. 

5  min.  55  sec. 

17  min.  31  sec.  (T) 

14  min.  0  sec. 

3  min.  40  sec. 

7  min.  7  sec. 

10  min.  25  sec. 

6  min.  25  sec. 

9  min.  52  sec. 

2  min.  56  sec. 

13  min.  45  sec.  (T) 

3  min.  38  sec. 

30  min.  0  sec.  (T) 

5  min.  9  sec. 

7  min.  22  sec.  (T) 

4  min.  2  sec. 

9  min.  20  sec.  (T) 

2  min.  58  sec. 

10  min.  29  sec. 

6  min.  11  sec. 

4  min.  20  sec. 

3  min.  10  sec. 

3  min.  0  sec. 

(T)  *  terminated 

before 

solution 

achieved 

3 


Average  Tine  for  Solution  When  Achieved 
(Terminated  Problem  Times  Not  Counted) 


9  minutes  39  seconds 


9  minutes  56  seconds 


4  minutes  57  seconds 


Average  Time  for  All  Problem  Attenpts 
(Terminated  Problem  Times  Counted) 


i 


15  minutes  22  seconds 


9  minutes  56  seconds 


4  minutes  57  seconds 


Figure  12 

Test  Scores  for  All  Students 
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Control  Group 

Evaluation  Groups 

No  CAI  Instruction 
No  Simulation 

CAI  Instruction  Only 
No  Simulation 

CAI  Instruction  and 
Simulation 

Number  of 

Students  16 

Tested 

10 

20 

Number  of 

Students  4 

Solving  Test 

Problem 

10 

20 

Percent  of 

Students  25% 

Solving  Test 

Problem 

100% 

100% 

Figure  13 

Surma  ry  of  Students  Successfully 
Completing  the  Test  Problem 


students  will  be  repairing.  However,  given  the  strong  trend  in  the  performance 
data  in  Figures  12  and  13,  it  also  appears  true  that  students  can  achieve  a 
high  degree  of  skill  on  the  simulation  to  give  them  confidence  as  they  move  to 
the  real  equipment,  increasing  the  effectiveness  of  real  equipment  time. 

Several  themes  ran  through  the  students'  comments  about  major  likes  and 
dislikes  of  the  system  as  uncovered  in  interviews  made  at  the  performance  test. 
The  most  popular  features  of  the  system  consistently  tended  to  be  the 
self-pacing,  the  clarity  and  understandability  of  the  message  due  to  its 
organizational  properties,  the  benefits  of  having  the  instructors,  the  realism 
of  the  problems,  and  the  general  inherent  interest  that  was  a  part  of  using  the 
system. 

Among  the  major  dislikes  of  the  system  were  the  lew  degree  of  audio  and  high 
degree  of  reading  (though  opinions  were  divided  on  this  issue  about  equally) 
programming  bugs,  the  length  of  the  introductory  and  demonstration  lessons,  and 
the  characteristics  of  certain  displays  (particularly  the  brightness  of  certain 
displays  and  the  "jitter”  associated  with  the  videodisc  picture) . 

Ccmnonly  recommended  improvements  in  the  system  included  additional  time  for 
instruction  to  take  place,  which  was  an  interest  expressed  by  several  students 
who  found  there  were  lessons  they  wanted  to  take  but  did  not  have  time  for,  the 
addition  of  audio,  the  elimination  of  program  bugs  and  the  addition  of  certain 
features  which  would  make  it  easier  for  the  student  to  have  mobility  into  and 
out  of  lessons.  Several  students  spontaneously  recommended  the  use  of  the 
system  at  tactical  sites  both  as  a  job  aid  and  as  an  instructional  and  practice 
source.  Sane  students  recommended  expansion  of  the  simulation  to  include  more 
than  the  present  circuits. 

The  reaction  to  the  hand-held  keypad  was  highly  positive,  and  no  negative 
conments  about  it  were  received,  with  the  exception  of  some  students  who 
indicated  that  the  temporary  lettering  on  their  keypad  keys  was  beginning  to 
wear  off.  Almost  universally  the  hand-held  keypad  was  preferred  over  keyboard, 
touch  screen,  or  light  pen,  all  of  which  were  specifically  mentioned.  Some 
problems  were  encountered  with  the  position  of  the  HELP  button.  Some  students 
mentioned  that  the  keys  on  the  keypad  were  in  themselves  a  training  aid  and  a 
reminder  of  functions  available  to  the  troubleshooter. 

Recommendations  for  addition  or  elimination  of  materials  from  the  system 
included  addition  of  materials  on  individual  end  items  in  the  I -HAWK  system. 
Some  students  in  Group  3  did  not  feel  that  the  Demonstration  lessons  had  a 
purpose*  However,  students  from  that  same  class  who  received  neither 
instruction  nor  simulation  found  themselves  unable  to  run  integrated  systems 
checks  in  order  to  solve  the  troubleshooting  problem.  Therefore,  we  conclude 
that  there  was  a  use  for  these  lessons,  but  one  not  perceived  by  the  students. 

Parts  of  the  system  indicated  as  most  beneficial  by  the  students  included  the 
close  relationship  between  lesson  objectives  and  lesson  oontent,  the  lessons  on 
the  use  4of  troubleshooting  aids  and  diagrams,  the  troubleshooting  exercises, 
and  the  lesson  on  troubleshooting  process.  The  simulation  (usually  referred  to 
as  "troubleshooting  problems")  was  most  often  mentioned  as  the  major  beneficial 
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feature  of  the  system.  Questions  7  through  23  of  the  questionnaire  asked  the 
student  to  rate  the  system  using  a  scale  of  1  to  5  on  several  factors.  A 
sunnary  of  the  results  of  those  ratings  is  presented  in  Figure  14  on  page  43. 

A  value  of  1  indicated  agreement  with  the  statement  and  a  value  of  5  indicated 
disagreement  with  the  statement.  Values  of  2,  3  and  4  indicated  moderate 
agreement  or  disagreement  or  neutral  feelings. 

Qiestionnaire  results  show  that  students  agreed  strongly  that  the  organization 
and  sequence  of  lessons  was  easy  to  follow,  that  they  would  reccnmend  these 
lessons  to  other  soldiers,  that  the  lesson  content  was  clear  and  easy  to 
follow,  that  the  keypad  was  well  marked  and  easy  to  use. 

'They  agreed  that  the  lessons  covered  about  the  right  amount  of  material,  that 
they  would  take  another  course  using  this  type  of  instruction,  that  they  would 
encourage  their  friends  to  take  this  type  of  instruction,  that  the  time  passed 
quickly  while  they  were  using  the  system,  that  the  content  of  lessons  was  at 
about  the  right  level  of  difficulty,  that  the  simulation  exercises  were  well 
done,  that  the  visual  displays  were  easy  to  read  and  understand,  that  the 
motion  sequences  helped  learning  and  that  the  simulation  exercises  helped 
improve  ability  to  troubleshoot  problems  with  the  I-HAWK  Integrated  Systems 
Checks. 

Ihe  students  tended  to  agree  with  the  statement  that  they  knew  where  they  were 
in  the  lessons  and  where  to  go  next,  but  were  not  as  strongly  positive  as  on 
other  items,  indicating  a  need  for  stronger  orientation  for  the  students 
concerning  the  lessons  to  be  taken  and  feedback  on  their  progress  through  the 
lessons.  Students  agreed  somewhat  with  the  statement  that  their  interest  in 
the  HANK  System  had  increased  because  of  these  lessons.  It  would  be  surprising 
if  this  item  had  received  too  strong  a  response,  since  24E10's  at  this  point  in 
their  training  are  considered  to  be  professionals  already  interested  in  their 
professional  skills  and  the  equipment  that  they  are  training  to  work  on. 

Students'  questionnaire  responses  to  question  18  show  they  appear  to  be  neutral 
on  the  quality  of  the  CAI  lessons.  Several  interview  and  other  questionnaire 
comments,  however,  indicate  that  students  enjoyed  the  lessons  and  felt  the 
content  in  them  was  appropriate  and  worthwhile.  An  interesting  trend  in 
attitudes  toward  the  lessons  is  apparent.  Groups  1  and  2  rated  the  CAI  lessons 
very  highly,  as  did  the  members  of  Group  3  of  lower  military  rank.  The  more 
experienced  members  of  Group  3,  however,  rated  them  very  negatively.  It  is 
possible  that  this  as  well  as  low  Group  3  ratings  on  other  items  are  due  to  the 
fact  that  the  DIS  materials  and  simulation  met  fewer  needs  because  those 
students  had  more  extensive  experience  with  the  I-HAWK  System  and  its 
procedures .  One  of  these  more  experienced  people  had  4  years  of  I-HAWK 
experience,  and  the  second  had  8.  It  is  to  be  expected  then  that  this 
simulated  training  would  hold  less  interest  and  benefit  for  them. 

Finally,  students  indicate  that  the  equipment  of  the  DIS  functioned  acceptably 
for  the  students ,  though  interview  and  other  questionnaire  results  indicate 
that  some  students  were  frustrated  by  the  problems  encountered  with  equipment 
performance.  The  indications  from  their  remarks  are  that  the  difficulties 
mainly  lay  in  program  bugs  or  program  features  which  made  manipulation  of  the 
program  difficult. 


Average  Rating 

1.58 

1.94 

1.68 

1.84 

1.61 

2.31 

1.68 

1.74 

2.05 

1.84 

2.26 

2.89 

2.00 

3.37 

1.79 

2.22 

2.16 


Statement 

7.  I  could  easily  follow  the  organization  of  the 
lessons. 

8.  Hie  lessons  covered  the  right  amount  of  material. 

9.  I  would  recommend  these  lessons  to  other  soldiers. 

10.  I  would  take  another  course  using  this  type  of 
instruction. 

11.  Hie  lesson  content  was  clear  and  easy  to  follow. 

12.  I  always  knew  where  I  was  in  the  lessons  and  where  I 
could  go  next. 

13.  Hie  keypad  was  well  marked  and  easy  to  use. 

14.  I  would  encourage  my  friends  to  take  this  training. 

15.  Hie  content  of  the  lessons  was  at  the  right  level  of 
difficulty. 

16.  Time  passed  quickly  when  I  was  using  the  new  training 
system. 

17.  My  interest  in  the  HAWK  System  has  increased  because 
of  these  lessons. 

18.  Hie  CAI  Lessons  were  excellently  done. 

19.  Hie  simulation  exercises  were  excellently  done. 

20.  Breakdowns  and  mechanical  problems  occurred  too 
frequently. 

21.  Hie  visual  displays  were  easy  to  see  and  understand. 

22.  Hie  motion  sequences  helped  my  learning. 

23.  Hie  simulation  exercises  really  helped  me  improve  my 
ability  to  troubleshoot  problems  with  the  HAWK 
integrated  system  check. 


Figure  14 

Suxrmary  of  Attitudes  Toward  the 
System  on  Several  Dimensions  from 
Questionnaire  Items 


Hie  interviews  conducted  during  system  evlauation  did  not  identify  new  areas  of 
information  about  the  system  and  its  acceptance  as  much  as  confirming  and 
proportioning  what  was  already  known.  One  interesting  finding  of  the 
interviews  was  that  students  tended  to  be  divided  on  most  issues  and  opinions 
expressed  toward  the  system.  Many  students  expressed  a  liking  and  appreciation 
for  the  demonstration  lessons.  Sane  students  did  not  express  as  strong  an 
appreciation  and  indicated  that  they  felt  the  lessons  were  too  long.  Most 
students  indicated  that  time  passed  quickly  and  that  they  did  not  experience 
fatigue.  However,  sane  students  indicated  that  the  brightness  of  the  screen, 
the  vividness  of  the  colors,  the  jitter  from  the  videodisc  image  and  the 
conflict  of  the  TV  image  with  their  glasses  caused  some  problems  of  eye  fatigue 
or  eye  strain.  This  was  alleviated  in  part  by  moving  the  students  back  from 
the  screen  to  view  it  from  a  further  distance.  The  one-on-one  trials  conducted 
before  the  Phase  II  evaluation  did  indicate  that  the  greater  distance  between 
the  student  and  the  TV  monitor  the  better  the  result  for  the  student  in  terms 
of  eye  fatigue  and  in  terms  of  providing  additional  working  space  for  the 
student. 

Students  tended  to  appreciate  the  "buddy  system"  that  was  used  during  the 
tests,  where  2  students  worked  at  one  terminal.  If  the  students  were  assigned 
with  a  buddy  who  worked  at  about  the  same  pace  and  with  the  same  learning 
style,  the  arrangement  appeared  to  be  beneficial,  but  some  incorpatabilities 
were  encountered  which  caused  students  to  remark  negatively  about  the  buddy 
system.  Probably  the  best  way  to  proceed  is  to  allow  students  to  choose  their 
own  buddies  when  pairing  up  for  instructional  purposes.  Sane  students 
preferred  the  buddy  system,  indicating  that  they  liked  the  conference  mode  of 
problem  solution. 

Most  students  expressed  appreciation  for  the  self-pacing  and  the  ability  to 
move  at  their  own  rate  of  speed  during  learning.  Many  students  expressed  the 
desire  to  take  lessons  which  were  available  on  the  system  but  which  were  not 
assigned  than.  Sane  students  appeared  to  be  so  wrapped  up  in  the  lessons  that 
when  break  time  came  they  found  the  break  a  nuisance  rather  than  a  benefit.  In 
one  case  during  the  evaluation,  students  were  taking  instruction  in  demon¬ 
stration  lessons  as  the  Space  Shuttle  landed.  Though  a  television  set  was  on 
in  the  instructional  area,  many  students  had  to  be  reminded  two  or  three  times 
to  watch  the  Space  Shuttle  land  instead  of  watching  their  lessons.  This 
interest  in  the  lessons  and  many  of  the  favorable  comments  about  the  lessons 
tempers  the  finding  of  other  data  that  the  lessons  may  have  been  boring  to  sane 
students.  Such  a  distribution  of  subjective  feelings  is  expected  in  any 
learning  environment.  The  data  indicates  that  students  who  received  no 
simulation  were  extremely  positive  toward  the  lessons.  However,  as  the 
simulation  was  introduced  to  students,  the  lessons  appeared  too  pale  by 
comparison  and  became  somewhat  lower  in  the  students'  priorities.  It  was  only 
at  this  point  that  students  began  to  feel  less  positively  toward  the  lessons. 
This  confirms  the  high  positive  valence  of  most  students  toward  the  simulation, 
and  we  would  expect  a  lower  rating  of  the  lessons  in  comparison  with  the 
simulation  for  those  students. 

Some  students  indicated  that  they  ielt  the  lessons  contained  too  much  reading. 
On  the  other  hand  several  students  indicated  that  they  appreciated  the  reading 
and  did  not  like  audio  presentations  for  instructional  purposes.  Several 


44 


students  mentioned  specifically  their  feeling  that  other  mediated  lessons 
conpared  poorly  with  the  videodisc  lessons  on  the  DIS. 


Several  students  indicated  that  they  had  no  discomfort  while  using  the  system. 
Other  findings#  however#  indicate  that  some  students  need  to  move  back 
from  the  monitor  die  to  potential  eye  strain  problems. 

Several  students  mentioned  the  usefulness  of  the  instructors  and  the 
friendliness  of  the  instructional  atmosphere  which  they  felt  was  more  conducive 
to  then  achieving  their  instructional  goals.  These  remarks  tend  to  confirm  the 
fact  that  the  preparatory  training  given  to  the  instructors  was  well  received 
by  the  instructors  and  implemented,  and  that  the  students  appreciated  the 
attitude  of  the  instructors. 

Though  several  students  expressed  the  interest  in  using  real  equipment#  several 
others  expressed  the  preference  of  the  videodisc  lessons  and  simulation  over 
real  equipment.  The  key  to  these  opposite  views  probably  lies  in  the  natural 
feeling  that  real  equipment  instruction  is  likely  to  be  more  effective.  Many 
intuitively  hold  that  position#  and  it  is  logical.  However#  those  students  who 
expressed  a  preference  for  videodisc  over  demonstrations#  may  have  been 
focusing  on  the  fact  that  demonstrations  using  real  equipment  are  often  staged 
improperly  or  fail  to  produce  the  desired  result  when  equipment  malfunctions. 
Students  in  the  classroom  instruction  group  for  this  evaluation,  with  few 
exceptions#  failed  to  get  hands-on  experience  due  to  large  group  size  and 
failed  to  see  most  of  the  demonstrations  when  the  equipment  was  incapable  due 
to  local  equipment  and  environmental  restrictions.  It  is  likely  that  the  best 
course  of  action  is  to  give  students  both  videodisc  simulator  and  real 
equipment  practice  but  to  ensure  that  the  practice  on  the  real  equipment  is  in 
fact  real#  working  practice.  The  best  plan#  of  course#  would  be  an  extended 
study  of  skill  development  profiles  using  the  simulator  and  using  real 
equipment. 


5. 5.3. 3  Phase  III  Results 

The  intent  of  Phase  III  was  to  try  out  lesson  and  simulation  naterials  with 
students  outside  MOS  24E10.  Though  formal  evaluation  procedures  were  called 
for  in  the  Evaluation  Plan  (i.e.#  observation  logs#  FITE  tests)#  they  were  not 
implemented  due  to  the  heavy  emphasis  placed  on  the  MOS  24E10  student 
evaluation.  The  instructional  naterials  were  used  with  several  groups  of  non- 
24E10  students  by  the  Ft.  Bliss  instructors,  but  no  formal  records  were  kept  of 
those  encounters#  so  no  results  are  reported  for  those  users. 


6.0  Conclusion  and  Recommendations 

The  high  degree  of  acceptance  of  the  DIS  installed  at  Ft.  Bliss  by  both 
students  and  instructors  and  the  high  levels  of  performance  of  students  who 
have  used  the  system  is  encouraging. 

The  original  system  goals  of  cost  reduction#  distributed  computing  power, 
applicaton  to  a  broad-spectrum  of  content  and  tasks#  high  visual  and  cognitive- 
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functional,  fidelity,  and  high  productivity  appear  to  have  been  met.  These 
concepts  embodied  in  the  DIS  have  been  demonstrated  to  be  potent  determiners  of 
student  achievement  and  potential  factors  for  reduction  of  future  training 
costs.  These  attainments  have  raised  questions  regarding  other  applications  of 
the  hardware  and  software  technology  used  in  the  DIS  both  within  and  outside  of 
the  NOS  24E10. 

We  recommend  four  immediate  areas  in  which  training  could  be  benefited  for  the 
I-HAWK  Missile  System  and  ways  within  each  of  those  areas  that  the  success  of 
the  current  program  can  be  used  as  a  beginning  point  for  further  training 
program  improvements. 

The  four  areas  of  improvement  are: 

1.  Expansion  of  Simulation  Scope 

2.  Improvement  in  the  Delivery  System 

3.  Expanded  Evaluation 

4.  Improved  Training  Base 

Each  of  these  areas  is  discussed  in  detail  below. 


RitpwntWi  jSiJUlflfclflO  top*-  The  present  capability  of  the  simulation  for 
improving  student  performance  should  be  expanded  from  its  present,  relatively 
limited  scope.  The  present  simulation  is  capable  of  simulating  problems 
occurring  during  the  first  step  of  the  Four-parameter  Auto  Designate  test  of 
the  HIPIR/LCHR  ISC  check.  Although  this  is  the  most  inclusive  of  the  ISC 
checks  covering  many  circuits,  it  is  only  one  portion  of  the  total  ISC  checks, 
all  of  which  must  be  trained.  In  the  Integrated  Systems  checks  there  are  about 
75  individual  tests,  all  of  which  could  be  simulated  with  a  suite  of  perhaps 
seven  simulations.  In  addition  to  the  seven  basic  integrated  systems  checks, 
there  are  also  communications  link-ups  between  the  I-HAWK  Missile  System  and 
higher  order  command  posts  for  which  simulation  of  circuits  is  appropriate.  At 
present,  training  on  those  circuits  is  below  desirable  levels  in  terms  of  the 
amount  of  practice  attainable  by  trainees  due  to  expensive  equipment  hook-ups 
required.  In  addition,  new  equipment  being  introduced  into  the  I-HAWK  Battery 
in  the  coming  years  will  present  difficult  training  challenges  to  the  DID,  and 
candidates  for  maintenance  simulations  include  the  Platoon  Command  Post  (PCP) , 
and  the  new  high-powered  illuminator  radar  (HIPIR)  scheduled  for  introduction 
within  the  next  year  or  two. 

A  second  recommended  area  for  expansion  of  the  simulation  capability  is  in  the 
area  of  operator  training.  Individual  and  team  training  exercises  are  possible 
for  procedures  training  using  the  system  as  it  is  presently  configured.  These 
may  include  simple  procedural  simulations  or  more  complicated  tactical 
simulations  which  we  believe  our  system  is  capable  of  administering. 

Dniiwry  fiyt— >-  The  delivery  system  as  presently  constituted  has 
been  adequate  in  many  respects  for  delivering  instruction  and  simulation 
exercises  to  students.  However,  we  recommend  the  following  improvements  to  the 
delivery  system  aimed  at  improving  the  ease  of  system  use  for  the  volume  of 
students  which  can  be  handled. 


First,  we  recommend  the  implementation  of  the  instruction  and  simulation 
programs  on  the  WICAT  150  Computer  rather  them  cm  the  networked  WICAT  100  DT  as 
the  system  is  presently  configured.  Individual  150  Model  computers  working  as 
independent  workstations  without  the  need  for  being  networked  are  capable  of 
administering  both  instruction  and  simulation  to  students.  The  150  comes 
housed  in  an  attractive  enclosure,  and  we  feel  it  will  be  more  easily 
maintained  in  the  future  years  than  the  present  100  DT  system  installed  at  Ft. 


Second,  we  recommend  the  acquisition  of  additional  terminals  to  be  installed  in 
the  classroom  at  Ft.  Bliss.  The  present  complement  of  four  terminals  has 
served  during  the  evaluation  for  both  individual-student  and  two-student  study 
mode.  Although  the  evaluation  found  the  two-student  study  mode  to  be 
successful  and  enjoyable  to  students,  some  students  find  this  arrangement 
difficult  to  work  in.  At  present,  classes  irust  be  divided  at  the  school.  The 
installation  of  additional  computers  at  the  Ft.  Bliss  classroom  would  allow 
entire  classes  to  be  instructed  at  once  without  the  "buddy"  requirement.  This 
would  also  allow  the  use  of  the  computers  by  classes  other  than  the  MOS  24E10 
classes.  Evaluation  results  have  shown  that  longer  exposure  to  the  system 
results  in  higher  levels  of  student  performance.  Though  this  result  is 
confined  to  the  somewhat  limited  conditions  of  the  evaluation,  we  feel  that 
additional  evaluations  of  the  system  will  demonstrate  that  this  is  a  consistent 
effect.  The  installation  of  additional  terminals  at  the  School  will  allow  a 
greater  amount  of  time  to  be  utilized  by  each  class  and  therefore  by 
implication  will  improve  the  levels  of  student  performance  attainable  at  the 
School. 

Third,  we  recommend  the  installation  of  a  printer  on  the  system  and  the 
provision  of  programs  not  presently  available  for  report  generation  in  printed 
form.  At  present,  reports  may  be  obtained  by  the  Instructor  or  Course  Manager 
at  the  storage  station  terminal  screw,  but  printed  reports  are  not  possible, 
and  no  printer  is  attached  to  the  storage  station.  The  storage  station 
computer  or  one  of  the  workstations  may  be  connected  directly. 

Bcpwrwted  Evaluation.  We  feel  that  the  evaluation  to  which  the  system  has  been 
subjected  thus  far  is  adequate  to  prove  its  effectiveness  but  does  not  give  a 
full  range  of  data  on  the  total  number  of  settings  and  applications  in  which 
the  simulation  and  instruction  may  be  effective.  We  recommend  the  following 
areas  for  extended  evaluation  activities. 

First,  we  recommend  that  the  extent  of  the  simulation  capability  for  training 
24E10  students  should  be  determined.  The  evaluation  time  constraints  have 
allowed  us  to  try  out  only  two  simulation  problems  with  the  students.  The 
simulation  is  capable  of  generating  literally  thousands  of  unique  problems. 
There  are  10  to  15  major  "types"  or  problems,  each  of  then  dealing  with  a 
different  set  of  circuitry  or  slightly  different  troubleshooting  steps.  These 
should  be  evaluated.  The  question  in  an  evaluation  of  this  sort  would  be  "How 
far  can  students  to  in  their  knowledge  of  troubleshooting  and  integrated 
systems  circuitry  using  the  DIS  I-HAWK  Simulation?"  We  believe  the  answer  to 
this  question  is  that  the  student  can  go  quite  far  in  mastering  the  integrated 
circuits,  particularly  when  the  simulation  is  used  in  tandem  with 


the  troubleshooting  exercise  lessons  as  has  been  the  case  during  the 
evaluation.  Me  believe  that  students  nay  be  sent  from  the  Ft.  Bliss  Air 
Defense  School  with  much  higher  levels  of  proficiency  than  is  currently  the 
case  and  that  this  would  require  only  a  natter  of  three  to  five  days  of 
additional  training  over  and  above  what  is  presently  given  the  students.  We 
believe  that  our  evaluation  showed  that  the  11/2  days  that  is  currently 
allocated  to  students  can  certainly  be  made  more  productive  than  it  has  been. 

We  recommend  that  the  evalution  completed  during  this  project  for  24E10  MOS 
students  be  enlarged  to  include  all  students  of  NOS  groups  currently  using  the 
system  at  the  school.  This  nay  include  24C  and  24G  students  as  well  as  24Rr 
Warrant  Officer ,  and  operator  students.  No  formal  data  gathering  took  place 
for  any  of  these  student  groups  although  the  reports  back  from  users  at  Ft. 
Bliss  indicate  that  several  groups  profited  from  using  the  system. 

In  addition  to  additional  MOS  group  evaluations,  we  would  recommend  the  use  of 
foreign  military  students  in  evaluations  as  well.  Reports  from  Ft.  Bliss 
indicate  that  non-U. S.  students  have  used  the  system  and  have  found  that  its 
low  audio  utilization  and  high  textual  orientation  is  congenial  with  their 
language  abilities  in  English.  We  would  recommend  the  evaluation  of  the 
system,  both  instruction  and  simulation,  with  non-U.S.  soldiers  and  would 
further  recommend  that  for  those  nationalities  not  capable  of  using  the  English 
version  of  the  materials,  non-English  versions  be  prepared  for  the  simulation. 
We  believe  this  could  be  done  in  a  relatively  easy  and  inexpensive  way,  but 
that  it  may  include  the  non-English  development  of  seme  of  the  troubleshooting 
exercises  and  introductory  lessons  as  well  as  the  simulation. 

We  reconmend  the  testing  of  the  simulation  and  instruction  in  the  field  as  a 
battery-attached  refresher  system.  Several  MOS  groups  are  likely  to  benefit 
from  the  lessons,  troubleshooting  exercises,  and  simulation  problems  the  system 
is  capable  of  presenting.  There  is  some  indication  in  fact  that  problems 
presented  by  the  simulator  are  more  appropriate  in  their  sophisticated  form 
for  more  advanced  and  experienced  students  than  are  found  at  the  24E10  course, 
if  this  first  iirpreesion  is  true,  then  we  would  be  inclined  to  say  that  the 
effectiveness  of  the  simulator  in  training  I -HAWK  maintenance  personnel  that 
has  been  observed  this  far  is  only  the  tip  of  a  training  iceberg  and  that  the 
potential  of  this  simulator  in  reducing  training  costs  and  inproving  maintainer 
performance  is  truly  remarkable  in  its  scope.  We  believe  that  an  evaluation  in 
the  field  would  detect  to  what  extent  and  with  what  groups  the  simulator  could 
have  some  effect. 

Finally,  we  reconmend  an  evaluation  of  the  factors  of  I -HAWK  training  using  the 
DIS.  Early  in  Phase  II,  a  cost  study  was  completed  showing  very  large  savings 
possible  through  simulations.  A  cost  model  unique  to  this  project  was  used  to 
forecast  several  levels  of  savings  possible  through  the  use  of  a  simulator, 
depending  on  the  simulator's  ratio  of  transfer  conpared  to  real  equipment. 

Data  may  be  easily  gathered  at  this  point  to  show  the  real  ratio  for  the  DIS 
simulation,  and  that  ratio  could  be  used  to  project  actually  attainable 
training  cost  reductions  attending  simulator  use. 
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The  instructional  materials  and  simulation  for  the  DIS  were  prepared  with  a 
relatively  minor  commitment  of  time  and  energy  to  24E10  job  task  analysis, 
training  resource  analysis  and  training  methods  analysis.  Me  recommend  that 
future  developments  for  either  instructional  lessons  or  simulations  be  prefaced 
with  appropriate  analyses  of  the  I-HAWK  Training  environment  of  the  total  group 
of  MDS's  involved  in  I-HAWK  Training.  Further,  other  Weapons  Systems  such  as 
ROLAND,  DIVAD,  and  PATRIOT  have  maintenance  training  requirements  that  can  be 
met  effectively  with  the  WIGAT  DIS.  We  believe  that  a  suite  of  analyses 
performed  appropriately  and  with  the  appropriate  tools  which  we  believe  were 
evolved  during  this  initial  development  effort  could  show  very  great  economies 
possible  in  I-HAWK  maintenance  training  as  well  as  providing  an  integrating 
view  of  those  areas  in  which  future  simulation  development  should  occur.  We 
recommend  the  following  types  of  analysis: 

e  Job  Task  Analysis.  We  recommend  the  conduct  of  a  job  task  analysis  for 
24E10  and  other  MDS's  trained  at  the  Ft.  Bliss  school.  If  these  analyses 
have  already  been  performed,  it  may  be  well  to  validate  those  task 
analyses  against  actual  field  requirements  as  has  been  done  in  several 
Army  communities  during  the  past  few  years,  including  the  Quartermaster 
School  and  more  recently  the  Signed  School.  We  believe  that  these 
analysis  programs  have  the  effect  of  improving  the  fit  between  school  and 
field  training  requirements  and  remove  from  the  school  undue  training 
loadB  for  training  which  can  be  accomplished  just  as  easily  in  the  field. 
At  the  same  time  they  insure  that  the  students  leaving  the  school  are 
ready  to  assume  those  duties  they  will  be  assigned  to  upon  reaching  the 
field.  Analytical  evidence  gathered  during  this  project  indicates  to  us 
that  at  present  there  is  not  a  good  fit  between  field  expectations  and 
graduate  capabilities  in  the  air  defense  community  and  particularly  in 
the  I-HAWK  oomiunity  for  maintainers. 

•  Study  of  Current  Training  and  Resources.  During  the  course  of  the 
present  contract,  WICAT  completed  a  study  of  the  costs  for  I-HAWK 
training  and  found  that  great  efficiencies  are  possible  using  the  small 
amount  of  technology  introduced  by  the  DIS.  We  recommend  a  study  of  the 
current  training  resources  and  programs  coupled  with  a  cost  calculation 
and  cost  tradeoff  study  like  that  conducted  for  this  project  to  extend 
across  multiple  MDS's  for  the  purpose  of  determining  areas  of  training 
which  are  sensitive  to  the  introduction  of  technology  for  the  purpose  of 
reducing  costs. 

•  Training  Systems  Recommendations .  We  believe  that  resulting  from  the 
above  analysis  and  cost  study  recommendations  there  should  be  a  set  of 
conclusions  concerning  I-HAWK  Training  and  possible  economies  which  are 
attainable  through  the  use  of  increased  levels  of  technology  in  the  Air 
Defense  School  curriculum.  These  recommendations  could  be  used  as  a 
foundation  on  which  to  build  an  improved  curriculum  and  general 
curriculum  practices  for  emerging  weapon  systems  such  as  the  Patriot. 


APPENDIX  A 
Early 
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Hie  Distributed  Instructional  System 


THE  DISTRIBUTED  INSTRUCTIONAL  SYSTEM 
FOR  I-HAWK  TRAINING 


The  Distributed  Instructional  System  (DJS)  is  a  powerful  computer-based  system 
designed  for  Computer-Assisted  Instruction  (CAI) ,  Computer-Managed  Instruction 
((MI) ,  and  two-  and  three-  dimensional  training  simulation.  The  system  also 
has  several  other  uses  in  management  of  business  and  training  information  and 
word  processing  which  are  not  covered  in  this  description. 

The  System 

The  structure  of  the  DIS  system  is  shown  in  Figure  1.  The  system's  computing 
power  is  distributed  among  individual,  interactive  terminals  called 
"workstations."  Each  workstation  has  its  own  powerful  microprocessor  computer, 
a  keyboard,  and  display  capability,  and  each  is  connected  to  a  central  "storage 
station."  The  storage  station  consists  of  a  computer  like  that  in  the 
workstations  and  a  20-45  megabyte  hard  disc  memory. 

Up  to  100  workstations  can  be  connected  to  a  storage  station  through  RS-232 
coaxial  cable  or  fiber  optic  links,  which  permit  extremely  rapid  transmission 
of  large  bodies  of  data  like  those  used  in  simulation  to  individual 
workstations.  In  the  distributed  instructional  system  concept  the  power  for 
confuting  is  located  at  the  individual  workstation  and  not  at  the  central 
storage  station.  Uhlike  monolithic  central  computer  systems,  there  is  no 
requirement  for  time  sharing  because  of  the  large  workstation  computer  power 
and  memory  and  because  of  the  rapid  data  transfer  capability  from  the  storage 
station  to  the  workstation. 

The  workstation  and  storage  station  controller  computers  are  based  on  the  new 
Motorola  MQ68000  microprocessor  chip,  which  will  be  augmented  by  256K  bytes  of 
RAM  memory.  The  MG68000  chip  is  a  powerful  microprocessor,  capable  of 
executing  two  million  instructions  per  second,  which  gives  it  roughly  the  power 
of  a  DEC  11/70  minicomputer. 

The  storage  station  and  its  attached  workstations  constitute  a  system  which  can 
stand  alone.  However,  to  facilitate  the  update  of  organizational  data  bases 
and  record  maintenance,  the  storage  station  may  be  hooked  through  conventional 
computer  linkages  to  virtually  any  host  computer.  This  link  can  be  made  at  low 
conputer  usage  periods  and  at  intervals  specified  by  the  user.  The  link  is  not 
necessary  for  the  system  to  function.  A  secondary  communication  and  update 
capability  for  programs  and  data  bases  is  provided  through  a  cartridge  tape 
drive  capability  at  the  storage  station. 

The  Economy  of  Distribution 

The  distribution  of  corrputering  power  to  individual  workstations  is  a  powerful 
concept  because  it  eliminates  the  inefficiencies  of  time  sharing.  It  also 
allows  for  a  computer-based  instructional  system  to  be  built  without  the 


Figure  1 

Structure  of  the  WICAT  Distributed 
Instructional  System 


initial  expense  of  a  large  central  computer  and  its  software.  Each  work¬ 
station  can  function  independently  and  is  self-sufficient ,  since  the  first 
workstation  purchase  is  in  a  storage  station  configuration.  Thus,  a  computer- 
based  system  nay  consist  of  one  terminal  or  several  terminals  with  the  oily 
incremental  expense  being  the  number  of  terminals  purchased. 

Training  Applications 

Although  the  DIS  is  a  computer  system  of  multiple  uses,  the  system  has  been 
optimized  for  training.  The  CAI,  CMI,  and  simulation  capabilities  of  the 
system  allow  it  to  save  training  costs.  The  DIS  workstation  has  been  developed 
to  include  the  latest  random  access  videodisc  player  technology,  which  is  used 
mainly  in  simulations,  but  which  is  also  highly  useful  in  certain  CAI 
applications. 

The  DIS  is  powerful  enough  to  execute  the  full  range  of  instruction  techniques. 

System  use  oy  instructors  and  writers  has  been  facilitated  by  the  creation  of 
the  Courseware  Design  System  (CDS) .  CDS  is  a  high-level  instructional  material 
authoring  language  which  allows  the  instructor  to  communicate  with  the  computer 
in  natural  language  (rather  than  computer  code)  to  create  instructional 
sequences  and  tests. 

Computer-managed  instruction  routines  are  capable  of  tracking  student 
instructional  parameters  and  progress.  They  are  also  capable  of  maintaining 
data  on  system  use  and  instructional  effectiveness.  These  are  extremely  useful 
tools  when  precise  knowledge  of  student  capability  and  progress  are  desired  on 
a  continual  basis.  The  CMI  capability  is  also  desirable  for  prescribing 
instructional  treatments  for  student  remediation.  CMI  acts  to  increase  the 
efficiency  of  the  training  system. 

The  two-dimensional  and  three-dimensional  simulation  capabilities  of  the  DIS 
are  among  its  most  important  training  features  because  of  their  tremendous 
potential  impact  on  the  overall  cost  of  training. 

For  training  in  the  use  of  complex  and  expensive  equipment,  or  when  the  skills 
begin  trained  are  dangerous,  simulation  is  capable  of  saving  large  amounts  of 
student  "hands-on"  time.  DIS  simulations  are  capable  of  paying  for  themselves 
through  savings  in  the  use  of  expensive  or  scarce  equipment.  Much  of  the 
equipment  can  then  be  made  free  for  operational  use  rather  than  for  training. 

Both  two-dimensional  and  three-dimensional  simulation  capabilities  are  possible 
using  the  distributed  instructional  system.  For  two-dimensional  simulations, 
student  interactions  with  the  simulation  program  take  place  through  the  CRT 
display  using  keyboard  or  light  pen.  For  three-dimensional  simulation  use,  the 
DIS  is  readily  interfaced  with  new  or  existing  training  equipment.  A  wide 
variety  of  simulations  are  possible  using  both  two-  and  three-dimensional 
simulations,  and  a  growing  body  of  training  developers  have  realized  the 
potential  power  of  such  simulations  and  the  attendant  cost  savings. 

Training  for,  the  I-BflWK  Missile  System  :-1 

Under  a  contract  presently  being  carried  out  for  the  U.S.  Defense  Advanced 
Research  Projects  Agency  (DARPA) ,  computer-assisted  instruction  and  two- 
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dimensional  simulations  are  being  created  for  training  of  U.S.  Army  students  in 
the  maintenance  of  the  I-HAWK  missile  system. 

The  scope  of  this  training  includes  the  isolation  and  correction  of 
malfunctions  occurring  during  the  Integrated  System  Checks.  The  instruction 
being  created  covers  the  normal  function  of  the  I-HAWK  system  during  the  checks 
and  the  use  of  manuals,  charts  and  diagrams,  consulted  during  fault  isolation. 

The  topic  of  instruction  is  the  function  of  the  system  during  the  Four- 
Parameter  ADP  Designate  portion  of  the  HIPIR/DCHR  check  and  the  use  of  system 
maintenance  manuals.  This  instruction  consists  of  six  CA.I  lessons  on  the  flow 
of  signals  during  the  check  and  the  use  of  system  manuals.  During  the  writing 
of  this  instruction  methods  have  been  developed  for  efficiently  authoring 
instruction  of  this  type.  Therefore,  extension  of  this  type  of  instruction  to 
to  other  ISC  checks  will  be  relatively  inexpensive. 

Slmflation 

A  two-dimensional  simulation  is  also  under  development.  It  includes  simulation 
of  problems  occurring  during  the  Four-Parameter  ADP  Designate  test.  The 
simulation  will  be  capable  of  creating  a  variety  of  problems  for  the  student, 
graded  over  the  easy-to-hard  range.  The  simulation  will  be  capable  of 
selecting  problems  at  the  appropriate  level  of  difficulty  for  the  student. 
During  simulation  problems,  the  student  will  interact  directly  with  the 
display,  making  tests,  observing  system  conditions,  replacing  components,  and 
performing  the  full  range  of  troubleshooting  procedures  as  if  interacting  with 
a  real  system.  At  the  end  of  a  troubleshooting  problem,  the  simulation  will 
determine  the  amount  of  time  and  money  spent  by  the  student  in  searching  for 
problems  and  replacing  faulty  parts. 

System  Coat 

dice  of  the  main  principles  guiding  the  design  of  the  DIS  has  been  cost 
minimization.  It  is  believed  that  the  DIS  provides  the  most  cost-effective 
means  of  providing  computer-assisted  instruction  and  simulation  for  most 
applications.  Since  the  system  is  constructed  one  terminal  at  a  time,  the  only 
incremental  costs  as  the  system  expands  are  the  costs  of  individual  terminals. 
The  system  user  does  not  pay  high  initial  costs  for  the  acquisition  of  costly 
computer  systems,  environments,  and  operators. 

The  DIS  system  does  not  incur  expenses  for  special  environments.  No  air 
conditioning  is  required,  and  there  are  no  special  power  requirements.  Since 
the  DIS  comes  housed  in  its  own  console,  there  is  no  additional  charge  for 
desks  or  carrels.  Headphones  are  provided  with  each  DIS  terminal  to  eliminate 
noise  interference  between  terminals. 

Conclusion 

This  summary  is  a  sampling  of  the  features  possible  using  the  DIS.  The 
system's  power  and  ability  to  be  used  in  different  configurations  allow  it  to 
handle  several  training  and  management  problems  simultaneously.  The  uses  of 
the  system  for  training,  and  especially  for  HAWK  training,  use  the  system's 
power  to  increase  the  overall  quality  of  instruction  while  reducing  its  costs. 
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APPENDIX  B 

System  Operating  Procedures 


SYSTEM  OPERATING  PROCEDURES 


This  section  provides  procedures  for  operating  the  I-HAWK  DIS.  The  system  has 
been  designed  to  operate  easily  and  require  no  programming  ability. 

These  procedures  assume  a  networked  system  properly  connected,  with  a  monitor 
attached  to  the  central  storage  station.  If  any  procedures  fail  to  work,  do 
not  attempt  to  fix  the  system  yourself.  Call  the  system  operator  to  help. 

The  procedures  outlined  are: 

System  Startup  (Page  B-l) 

System  Turn-off  (Page  B-2) 

Workstation  Reboot  (Page  B^2) 

Storage  Station  Reboot  (Page  B-3) 

Disc  Side  Change  (Page  B-3) 

Student  Registration/Deletion/Change  (Page  B-4) 

Report  Viewing  (Page  B-4) 

Exam  Maintenance  (Page  B-4) 


SYSTEM  STARTUP 

System  startup  has  been  automated  to  the  point  where  it  consists  mainly  of 
powering  up  individual  system  components  in  a  given  order.  Follow  this 
procedure  for  system  startup: 

1.  Turn  on  each  workstation  TV  monitor  by  pulling  the  on-off  knob  out. 

The  knob  is  located  below  and  to  the  left  of  the  channel  selector. 

Turn  the  volume  fully  counterclockwise  using  the  same  knob. 

2.  Turn  on  each  workstation  computer. 

(a)  Insure  that  the  power  connector  is  plugged  into  a  wall  socket,  that 
the  computer  is  plugged  into  the  power  connector,  and  that  the 
power  connector  is  turned  on.  Whan  the  connector  is  on,  the  orange 
lanp  will  glow  on  the  connector . 

(b)  Push  the  top  half  of  the  conputer's  power  rocker  switch  to  ON.  The 
rocker  switch  is  located  at  the  upper  left  corner  of  the  rear 
computer  panel  as  you  face  the  computer.  As  you  turn  cm  the 
computer  you  will  hear  a  fan  inside  begin  to  turn. 

3.  Turn  on  each  workstation  videodisc  player. 

(a)  Insure  a  videodisc  is  loaded  in  each  machine. 

(b)  Insure  that  the  player  cover  is  closed  tightly. 

(c)  Press  the  key  labelled  POWER  until  it  catches  in  the  ON  position. 

(d)  Press  the  key  labelled  PLAY  and  watch  to  insure  that  the  disc  moves 
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under  the  play  area. 

(e)  After  counting  to  15  slowly,  press  the  STOP  key  until  the  red  lamp 
above  the  key  remains  on. 

4.  Turn  on  the  storage  station  monitor  using  the  rocker  switch  at  the 
bottom  right  rear  of  the  monitor  case.  You  will  hear  a  "beep"  sound 
when  it  receives  power.  Wait  until  the  square  white  cursor  appears  on 
the  screen  before  proceeding. 

5.  Turn  on  the  storage  station  using  the  rocker  switch  on  the  front  left 
panel  of  the  storage  station.  The  red  light  on  the  front  panel  will 
illuminate,  and  the  storage  station  fan  will  begin  to  hum.  Writing 
will  begin  to  appear  on  the  monitor  screen. 

At  this  point,  wait  until  the  TV  monitor  at  each  of  the  workstations  presents  a 
text  message.  This  may  take  up  to  ten  minutes. 


SYSTEM  TURNOFF 

Turn  off  system  components  only  after  all  workstations  are  logged  off  to  avoid 
loss  of  student  data. 

Turn  off  system  components  in  this  order: 

1.  Storage  station  computer. 

2.  Storage  station  monitor. 

3.  Workstation  videodisc  player. 

(a)  Press  the  key  marked  REJECT  and  wait  for  the  disc  to  come  to  a  step 
before  proceeding. 

(b)  Press  the  key  marked  FCWER  until  it  cones  out  all  the  way  and  the 
red  lamp  above  it  extinguishes. 

4.  Workstation  conputer. 

%  5.  Workstation  TV  monitor. 

WORKSTATION  REBOOT 

When  a  workstation  hangs,  it  may  be  necessary  to  reboot.  Use  this  procedure 
for  rebooting: 

1.  Press  the  flat,  flush-mounted  conputer  reset  switch  and  release  it.  it 
is  placed  to  the  right  of  the  workstation  computer's  power  switch  as 
you  face  the  computer. 

2.  Wait  up  to  five  minutes.  The  text  message  should  appear  on  the 
workstation  TV  monitor. 


STORAGE  STATION  REBOOT 


When  all  workstations  are  hung  and  it  appears  to  be  a  storage  station  problem, 
you  may  have  to  reboot.  Use  this  procedure  for  rebooting: 

1.  Press  the  flat,  flush  mounted  computer  reset  switch  and  release  it.  It 
is  mounted  beside  the  computer  power  rocker  switch. 

2.  Wait  until  the  storage  station  asks  you  for  the  date.  Press  RETURN  on 
the  storage  station  monitor  keyboard  until  you  see  a  series  of  ">" 
symbols  in  a  vertical  line  on  the  screen. 

3.  Reboot  all  workstations. 


DISC  SIDE  CHANGE 

When  the  workstation  TV  monitor  displays  a  message  requesting  a  change  of  the 
videodisc  to  a  new  side,  follow  this  procedure: 

1.  Press  the  key  marked  REJECT  and  wait  until  the  disc  comes  to  a  full 
stop. 

2.  Press  the  key  marked  COVER  RELEASE.  The  player  lid  should  pop  open 
slightly. 

3.  Lift  the  player  lid  fully  open. 

4.  On  the  central  disc  spindle,  grasp  the  spindle  at  the  indented  spot 
half-way  down  the  shaft.  Pull  up  until  the  foot  holding  the  disc  in 
place  lifts  and  you  hear  a  click. 

5.  Grasp  the  disc  at  its  left  edge  and  lift  it  from  the  spindle  carefully. 

6.  Turn  the  disc  over  and  replace  it  on  the  spindle  with  the  left  side 
higher  than  the  right  by  about  11/2  inches  until  the  disc  lays  flat  on 
the  spindle  bottom. 

7.  Pinch  the  top  of  the  spindle  as  narked  until  the  disc  holder  foot 
clicks  into  place. 

8.  Close  the  player  cover  until  it  clicks  securely  shut. 

9.  Press  the  key  marked  PLAY.  Watch  until  the  disc  moves  in. 

10.  Press  the  key  marked  STOP  until  the  red  lamp  above  it  remains  lit. 

11.  Press  ENTER  on  the  student's  keypad.  A  video  picture  should  appear 
after  a  very  short  time. 
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STUDENT  RBGISTRATIC^DELETION/CHANGE 

To  register  or  delete  a  student  or  modify  student  registration  data,  follow 
this  procedure: 

1.  Insure  that  the  storage  station  monitor  is  connected  to  the  storage 
station  computer. 

2.  Press  RETURN  several  tines.  Several  ">"  symbols  should  appear. 

3.  Type  /TCPKOG  and  press  RETURN.  A  menu  should  appear. 

4.  Select  "Registration”  from  the  menu  by  following  the  instructions  on 
the  screen. 

From  this  point,  full  instructions  are  provided  at  the  monitor  by  the  program. 
Follow  those  instructions.  When  you  are  finished,  be  sure  to  "QUIT"  the 
program  from  the  menu. 


REPORT  VIEWING 

To  view  system  reports,  follow  this  procedure: 

1.  Insure  that  the  storage  station  monitor  is  connected  to  the  storage 
station  conputer. 

2.  Press  RETURN  several  times.  Several  ”>”  symbols  should  appear. 

3.  Type  /TCPROG  and  press  RETURN.  A  menu  should  appear. 

4.  Select  "Reports"  from  the  menu  by  following  the  instructions  on  the 
screen. 

From  this  point,  full  instructions  are  provided  at  the  monitor  by  the  program. 
Follow  those  instructions.  When  you  are  finished,  be  sure  to  “QUIT"  the 
program  from  the  menu. 


EXAM  MAINTENANCE 

The  Exam  Maintenance  program  is  designed  to  allow  the  instructor  to  control  the 
content  of  exams.  There  are  two  areas  involved  in  exam  maintenance: 

1.  The  controlling  factors  in  the  exam: 

-  The  number  of  questions  to  be  asked 

-  Percentage  correct  to  pass 

-  The  exam  passcode  (entered  by  the  instructor) 

-  The  maximum  time  (in  minutes)  allowed  for  the  student  to  take  the 


2.  Maintaining  question  content: 

-  Adding  questions 

-  Deleting  questions 

-  Changing  questions 

-  Viewing  questions  for  review  purposes 

The  Exam  Maintenance  routine  is  accessed  at  a  workstation  by  an  instructor 
logged-on  under  either  an  instructor  or  a  manager  user  number. 

At  the  System  Menu,  NEXT  is  pressed,  which  brings  up  page  2  of  the  menu.  "Exam 
Maintenance"  should  be  selected  at  this  point,  which  calls  the  maintenance 
program. 

Upon  entering  the  program,  the  instructor  sees  the  following  menu: 

1.  View/change  exam  size  or  criteria 

2.  Change  exam  question 

3.  Add  exam  question 

4.  Delete  exam  question 

5.  View  exam  question 

6.  Quit 

Option  one  (1)  allows  the  instructor  to  change  or  set  the: 

-  Number  of  questions  to  be  asked:  A  maximum  of  50  questions  may  be 
given  to  the  student  because  the  maximum  number  of  questions  in  the 
pool  is  50.  The  minimum  number  allowed  is  1. 

-  Percentage  correct  to  pass:  This  field  will  be  expecting  a  value 
between  1  and  100.  No  decimal  points  will  be  allowed  and  no  percent 
signs  (%)  will  be  necessary.  A  "50"  typed  in  will  mean  50%,  a  "75" 
signs  (%)  will  be  necessary.  A  "50"  typed  in  will  mean  50%,  a  "75” 
will  mean  75%.  This  percentage  is  the  percentage  of  those  questions 
given  to  the  student  that  must  be  answered  correctly  in  order  to  pass 
the  exam. 

-  Exam  passcode:  Before  the  student  can  take  an  exam,  the  instructor 
must  enter  a  4-digit  passcode.  (No  alphabetic  characters  are  allowed.) 
Each  exam  may  have  a  different  passcode,  or  they  may  all  be  the  same. 

-  Maximum  time:  This  is  the  maximum  time,  in  minutes,  the  student  will 
be  allowed  to  spend  on  an  exam.  It  must  be  a  value  greater  than  zero. 
At  the  end  of  this  time  the  exam  will  be  terminated  and  scored.  If  the 
student  has  enough  correct  answers  to  pass,  even  though  he  ran  out  of 
time,  he  will  still  pass.  If  he  does  NOT  have  enough  correct  answers 
by  this  time,  he  will  NOT  pass  the  exam. 


Option  one  also  displays  the  number  of  questions  currently  existing  in  the 
lesson.  Instructions  on  the  screen  direct  the  instructor  how  to  respond. 

Maintaining  questions  is  done  using  options  2-5  of  this  main  menu: 

1.  View/change  exam  size  or  criterion 

2.  Change  exam  question 

3.  Add  exam  question 

4.  Delete  exam  question 

5.  View  exam  question 

6.  Quit 

Upon  duosing  to  DELETE  a  question,  the  instructor  is  shown  the  text  of  the 
question  and  must  confirm  the  delete  with  a  "Y"  (Yes)  or  "N"  (Mo) . 

When  VIEWing  a  question,  the  instructor  is  shown  what  the  question  currently 
looks  like,  but  the  question  cannot  be  changed,  added  to,  or  deleted  in  this 
mode. 

lb  CHANGE  or  ADD  a  question,  the  instructor  follows  the  same  process,  the  only 
difference  being  that  a  question  to  be  added  has  no  information  in  it,  while  a 
question  to  be  changed  has  the  existing  information  in  it. 

For  adding  and  changing  lessons,  after  the  selection  of  the  lesson  number  and 
question  number  the  instructor  is  asked  to  press  RETURN  and  then  add  or  change 
the  text  of  the  question.  On  pressing  RETORN  he  either  sees  a  blank  screen 
(if  adding)  or  the  text  of  the  question  (if  changing) .  In  the  top  left  comer 
of  the  screen  will  be  a  cursor  (_)  that  signifies  where  he  will  enter  text.  He 
may  then  move  the  marker  anywhere  on  the  screen  and  type  the  question  just  as 
he  wishes  the  user  to  see  it.  Following  are  certain  commands  that  may  be  used 
while  editing: 

*A  -  Move  to  the  front  of  the  line.* 

*B  -  Delete  previous  word. 

~S  -  Delete  current  word. 

*D  -  Delete  to  end  of  the  line  from 
*F  -  Move  to  beginning  of  next  word. 

~G  -  Move  to  end  of  the  line. 

*0  -  Move  to  the  beginning  of  the  next  line. 

*R  ~  Move  to  the  beginning  of  the  previous  word  on  the  same  line. 

*T  -  Move  to  the  top  of  the  screen. 

*0  -  Delete  from  to  the  front  of  the  line. 

*V  -  Delete  the  current  character. 

*The  "'‘"symbol  signifies  the  control  key  (labelled  CTRL  on  the  keyboard) . 
Whenever  the  symbol  is  encountered,  it  means  to  strike  the  following  key  while 
HOLDING  DOWN  the  control  key. 

The  arrow  keys  and  delete  (DEL)  key  are  also  functional. 
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SYSTEM  MAINTENANCE  PROCEDURES 


There  are  no  system  maintenance  procedures  advised  to  be  performed  by  system 
users.  The  system  equipment  has  been  designed  or  selected  for  low  maintenance 
requirements.  Dust  on  the  fan  cover  opening  may  be  removed  with  a  vacuum 
periodically  to  inprove  air  flow.  Floppy  disk  drive  heads  and  tape  drive  heads 
should  not  be  serviced  by  users,  even  for  cleaning  purposes. 
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EVALUATION  AND  IMPLEMENTATION  PLAN 
FOR  THE  DIS  -  TASK  2  EVALUATION 


i.  xnttadUBtJLflD 

Hie  purpose  of  this  plan  is  to  specify  the  tasks  and  schedules  associated  with 
the  evaluation  of  the  equipment  and  software  system  prepared  for  the  use  of  the 
U.S.  Army  Air  Defense  School  (USAADS)  for  the  training  of  HAWK  missile 
maintenance  personnel. 

TT  fhMrt-NM  ftasraHfWil  Evaluation 

A.  Objective:  The  objective  of  this  evaluation  is  to  assess  the  reliability 
of  DIS  hardware  and  software. 

B.  Schedule:  The  evaluation  will  take  place  during  a  one-week  period  between 
15  July  and  15  September  1981.  Exact  dates  will  depend  upon  the 
availability  of  Army  personnel. 

C.  Method:  Die  evaluation  consists  of  four  users  exercising  the  system  for 
eight  hours  per  day  using  the  networked  DIS  software  and  at  least  one 
instructional  lesson. 

D.  Questions:  Questions  to  be  answered  during  this  evaluation  include: 

Harttorp 

•  Do  WICAT  systems  diagnostics  function  reliably  during  workstation 
power-up? 

•  Does  the  storage  station  successfully  boot  from  the  tape  drive? 

•  Does  the  storage  station  dunp  data  to  the  tape  and  successfully 
retrieve  it? 

Software 

•  Does  the  instructional  software  perform  without  failure  through  a 
specified  test  period? 

•  Does  the  system  software  perform  without  failure  through  a  specified 
test  period? 

•  Does  the  system  software  survive  specified  rigor  tests? 

—  Simultaneous  request  tests 
—  Volume  request  tests 
—  Simultaneous  station  use  tests 
—  Station  casualty  tests 
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In  the  event  that  24E  HAWK  students  ace  used  as  testers,  additional  evaluation 
data  may  be  gathered  on  the  following  instruction-related  questions. 

e  Are  the  formats  and  patterns  of  interaction  for  the  lesson  easy  to  use, 
appropriate  to  the  audience? 

e  Is  the  content  level  of  the  lesson  appropriate  to  the  audience? 
e  Are  students  able  to  learn  and  achieve  mastery  from  the  instruction? 

E.  Resources:  During  the  evaluation  week,  four  Amy  testers  will  be 
required. 

Free  access  to  the  testing  area  before  and  after  testing  hours  will  be 
necessary  to  allow  adjustment  and  programs. 

TWo  WICAT  personnel  will  be  present  during  the  evaluation  period  to  provide 
technical  support  for  the  effort,  observe  the  tests,  and  gather  evaluation 
data.  The  USAADS  Evaluation  Coordinator  will  also  be  required  to  interface 
with  tester  personnel,  manage  facilities,  and  provide  direction  for  USAADS 
during  the  evaluation.  The  coordinator  should  be  present  during  the  evaluation 
to  assure  USAADS  that  the  data  is  being  properly  gathered  and  that  tests  are 
being  properly  run. 

F.  Revisions:  Revisions  to  hardware  and  system  software  which  are  required 
to  bring  the  system  operations  to  within  the  operating  criteria  will  be 
made  during  the  evaluation  period.  Incidental  data  on  instruction  will  be 
applied  to  revision  of  the  CAI  lessons. 


APPENDIX  E 

Test  Problem  Presentation 


The  speech  used  for  presentation  of  the  problem  to  test  subjects  is  given 
below. 

The  Army  tester  would  say: 

"This  will  be  a  test  of  your  troubleshooting  ability.  I  will  act  as  a 
BCC  connected  in  a  functioning  I-HAWK  system.  I  will  present  to  you  a 
problem  symptom  found  while  running  the  4-Parameter  Auto  Designate  Test  of 
the  High  Bower/Launcher  check.  You  can  request  information  from  me  about 
switch  settings,  indications,  tests,  or  measurements,  and  I  will  respond 
by  giving  you  any  information  normally  available  to  a  troubleshooter  with 
tools.  You  must  isolate  the  fault  to  a  component ,  replace  the  component, 
and  re-run  the  4-Parameter  test  successfully.  This  will  be  a  timed  test. 
Here  is  the  synptcm.  While  running  the  test,  the  test  goes  all  the  way  to 
completion,  but  the  SIGNAL  STRENGTH  meter  on  the  PC  panel  fails  to  show 


APPENDIX  F 

Sample  Student  Attitudinal  Questionnaire 


HAWK  VIDEODISC  TRAINING  QUESTIONNAIRE 


Directions:  This  questionnaire  asks  for  your  comments,  ratings  and  personal 
reactions  to  the  HAWK  videodisc  training  system  and  lessons.  Your 
honest  comments  and  reactions  will  help  us  to  revise  and  inprove 
the  HAWK  training  system  for  future  groups  of  soldiers.  Your 
responses  will  be  kept  anonymous  and  confidential. 


I.  Biographical  Information 


1.  _ Instructor  _ Student 

2.  Current  MOS _ 

3.  RANK _ 


4. 

5. 


Sex: 


Age: _ 


6.  Years  in  Armed  Services 


7.  Which  of  the  following  types  of  instruction  have  you  had  experience 
with  and  which  do  you  like  best?  Place  a  check  (  )  in  the  first 
column  for  all  types  of  instruction  you  have  experienced.  Then  rank 
each  of  your  responses  in  order  of  your  preference.  (1-first, 
2-second,  3-third,  etc.) 


Exper.  Rank 

1.  _  _ 

2.  _  _ 

3.  _  _ 

4.  _  _ 

5.  _  _ 

6.  _  _ 

7.  _  _ 

8.  _  _ 

9.  _  _ 

10.  _  _ 


computers 

videodisc 

intelligent  videodisc 
snail  group  discussion 
TV 

films 

individual  study 
sinulation 
classroom  lecture 
others  (Specify) _ 
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II.  General  Questions 


1.  What  did  you  particularly  like  about  the  HAWK  videodisc  training 
system  and  lessons? 


CAI  Lessons 


Simulation 


2.  What  did  you  dislike  about  the  HAWK  videodisc  training  system  and 
lessons? 


CAI  Lessons 


Simulation 


3.  What  improvements  would  you  recommend  for  the  HAWK  videodisc  training 
system  and  lessons? 


CAI  Lessens 


Simulation 


4.  What  are  your  personal  reactions  to  the  keypad  for  entering  responses? 


.  IV'  **  O  C'  V.’./- 


5,  Are  there  any  materials  which  should  be  added  to  or  eliminated  from 
the  videodisc  training? 

Metfiaia  *■*  h»  aAted  Materials  tfl  he  EUninated 


6.  Which  lessons  or  parts  of  the  CAI  instruction  and  Sinulation  training 
were  most  beneficial? 

CM  Instruction  SiilUlfitlon 


For  questions  7-24  circle  your  response  using 
the  scale  on  the  right. 


7.  I  could  easily  follow  the  organization  and 
sequence  of  the  lessons. 

8.  The  lessons  covered  the  right  amount  of 
material. 

9.  I  would  recommend  these  lessons  to  other 
soldiers. 

10.  I  would  take  another  course  using  this  type 
of  instruction. 
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11.  The  lesson  content  was  clear  and  easy  to  follow 

12.  I  always  knew  where  I  was  in  the  lessons  and 
where  I  could  go  next. 

13.  The  keypad  was  well  marked  and  easy  to  use. 

14.  I  would  encourage  my  friends  to  take  this 
training. 

15.  The  content  of  the  lessons  was  at  the  right 
level  of  difficulty. 

16.  Tine  passed  quickly  when  I  was  using  the  new 
training  system. 

17.  My  interest  in  the  HAWK  system  has  increased 
because  of  these  lessons. 

18.  The  CAI  lessons  were  excellently  done. 

19.  The  simulation  exercises  were  excellently  done. 

20.  Breakdowns  and  mechanical  problems  occurred 
too  frequently. 

21.  The  visual  displays  were  easy  to  see  and 
understand. 

22.  The  motion  sequences  helped  my  learning. 

23.  The  simulation  exercises  really  helped  ne 
improve  my  ability  to  troubleshoot  problems 
with  the  HBWK  integrated  system  check. 

24.  How  was  the  HAWK  videodisc  training  different 
from  other  learning  and  training  experiences 
you  have  had? 


